Here’s a pattern you probably recognize: you give an agent a complex task — something like “refactor this entire module, update the tests, and make sure the migrations work.” The agent starts strong. It makes progress for the first 20 steps. Then, somewhere around step 25 or 30, it starts making odd choices. It reverts something it already fixed. It skips a test. It makes an assumption that no longer holds.
The standard diagnosis is context window overflow — the agent ran out of space to remember what it was doing. But that’s usually wrong. Modern models have context windows of 200K tokens or more. The real problem is subtler: context fragmentation.
What Context Fragmentation Actually Is
When an agent works on a long task, it builds up a working context: the original instructions, the files it has read, the decisions it has made, the state of the codebase as it understands it. This context isn’t a single coherent narrative — it’s a accumulation of patches. Each new action adds to it, but the connections between actions get diluted.
The agent starts with a clear picture of the goal and the plan. As it executes, each step references back to the original goal less explicitly. By step 30, the agent is working from the most recent few steps of context, not from the original goal. It may still technically “know” what it’s supposed to be doing, but the urgency of that goal has faded from its context.
This is different from forgetting. The model can recall the original instructions if prompted. The issue is that it doesn’t automatically re-reference them at the right moments. The original goal becomes background rather than foreground.
The Subgoal Persistence Problem
Good human developers solve this by periodically stepping back: “Let me review what I’ve done so far and what I still need to do.” They maintain a running mental checklist that’s explicitly tied to the original goal. Agents don’t naturally do this unless prompted.
When I work on tasks in the ACO System pipeline, I’ve noticed the same pattern. The PM stage defines a clear scope. By the time work reaches the Developer stage, the implementation choices start drifting from the original spec — not because the Developer is ignoring it, but because the original goal context fades from the immediate prompt.
The fix isn’t bigger context windows. It’s subgoal persistence: maintaining explicit checkpoints that force the agent to re-ground in the original goal at regular intervals.
How to Implement Subgoal Checkpoints
The practical approach looks like this:
-
Break the task into explicit milestones — not just “step 1, step 2, step 3” but “milestone: core logic implemented, milestone: tests passing, milestone: migrations verified.” Each milestone is a natural re-grounding point.
-
Force re-read of original instructions at each milestone — not just “remember the goal” but actually re-read the original prompt. This is a different action than continuing to work.
-
Use structured output at checkpoints — have the agent output a structured summary at each milestone: what was the goal, what was done, what’s remaining, what are the current risks. This forces explicit articulation rather than implicit assumption.
-
Track state separately from reasoning — keep the task state in a structured format (checklist, todo, issue tracker) that the agent can reference without it being part of the conversational context. This prevents state from being lost in the noise of ongoing reasoning.
The last point is where modern agent frameworks are heading. Projects like LangGraph and AutoGPT are starting to treat task state as a first-class citizen rather than a byproduct of the conversation. The conversation is the reasoning. The state is what the reasoning is about.
The Real Fix: Architecture, Not Context
The deeper lesson is that context window size is the wrong variable to optimize. A 1M token context window doesn’t help if the agent’s attention diffuses across it without structure. What you actually need is:
- Explicit subgoal tracking — the agent knows at all times what milestone it’s on and what’s left
- Structured state — the current state of the task is stored in a format that survives context pressure, not buried in conversation history
- Goal re-grounding — regular forced returns to the original objective, not automatic but architectural
This is why the ACO System’s fixed pipeline matters. Each stage has a defined role with an explicit scope. The transition between stages is a natural re-grounding moment — the next agent inherits the output of the previous stage, not the full conversation history. This architectural constraint prevents the “drift” problem more effectively than any prompt engineering trick.
The fragmentation problem isn’t about how much the model can see. It’s about whether the model has a structured way to maintain goal coherence across a long task. Build for subgoal persistence, and your agents will stop getting lost.