Context Rot Is Killing Your Long-Running Agent: A Compaction Playbook That Survives 100k+ Tokens

Your long-running agent is not failing because it ran out of tokens. It is failing because its accuracy quietly degrades as the context grows, an effect now widely called context rot. The 2026 Chroma study across 18 models showed the same pattern everywhere: as you pack more into the context, the model gets worse at using any single part of it, even when the window is nowhere near full. The fix is not a bigger model or a longer window. It is context engineering: deciding what stays in the live context, when to compact it, and what to push out to external memory. This is the playbook I use to keep agents reliable past 100k tokens.

What is context rot, and why does it kill long-running agents?

Context rot is the measurable drop in a model's accuracy as the amount of context you give it grows, independent of how much room is left in the window. The longer your agent runs, the more tool outputs, intermediate reasoning, and conversation history it accumulates, and the worse it gets at attending to the parts that matter.

Two failure modes stack on top of each other in a long-running agent:

• Lost-in-the-middle. Models attend strongly to the beginning and end of their context and weakly to the middle. A critical instruction buried at turn 30 of an 80-turn trajectory effectively disappears.


• Distractor accumulation. Every tool call dumps more text in. Old, stale, or near-duplicate content competes for attention with the few tokens that actually drive the next decision.

Does a bigger context window fix context rot?

No. A bigger window buys you more turns before collapse, but it does not change the curve. The accuracy drop scales with how much you actually put in the context, not with how much capacity remains.

I have watched teams treat a 200k or 1M token window as a license to never prune anything. That is the trap. You hit the same degradation at 120k that you would have hit at 60k with a smaller model, just later and with a larger bill. The window size sets your ceiling. Context engineering sets how close to the ceiling you can run before quality falls apart.

The mental model I keep: treat the context window like working memory, not a hard drive. Working memory is for what you need right now. Everything durable belongs somewhere else and gets pulled back in on demand.

Trigger-based vs reactive compaction: which should I use?

Use trigger-based compaction. Reactive compaction, where you only summarize once you are about to overflow the window, means the model spends its final and most important turns operating inside an already-degraded context. That is the worst possible moment to compact.

Trigger-based compaction fires on an explicit condition before quality degrades, so the agent never works in a rotted window. Here is the difference laid out:

def should_compact(state) -> bool:
    over_budget = state.live_tokens > 0.5 * state.window_size
    subtask_done = state.last_event == "subtask_complete"
    huge_tool_output = state.last_tool_tokens > 8_000
    return over_budget or subtask_done or huge_tool_output

[ ANCHOR ]   immutable: goal, hard constraints, output contract, key IDs
[ SUMMARY ]  rolling: compressed history of completed work + decisions
[ ACTIVE ]   recent: last N turns + most recent tool outputs, verbatim

You are compacting an agent's working context. 
Do NOT touch or restate the ANCHOR; it is preserved separately.

Rewrite the SUMMARY so it captures, in under 400 tokens:
- Decisions made and WHY (not just what)
- Facts discovered that future steps depend on (IDs, paths, values)
- Dead ends already tried, so they are not repeated
- Open questions still unresolved

Drop: raw tool dumps, exploratory reasoning, anything superseded.
Preserve exact identifiers verbatim. Never invent state.

[context status] live: 41k / 200k tokens (~20%). Budget healthy.
You may continue exploring.

[context status] live: 108k / 200k tokens (~54%). Budget tight.
Wrap up the current sub-task, persist any durable findings to memory,
and prepare for compaction. Avoid starting new wide searches.

Concretely: when a tool returns a 12k-token document, I do not keep it in context. I write it to a store, keep a one-line pointer plus the few extracted facts I need, and retrieve the full text again only if a later step requires it. The agent operates on a compact index of what it knows and pulls detail on demand. That is retrieval, applied to the agent's own working state rather than to a static corpus.

This is also where memory framework choice matters, because retrieval latency and recall accuracy become part of your agent loop. I went deep on the tradeoffs in my benchmark of Mem0, Letta, Zep, and LangMem ↗, and the headline is that the store you pick directly affects how aggressively you can compact without losing fidelity. A good external memory layer lets you run the live context lean, which is the whole point.

One caution that ties back to security: anything you write to external memory and later retrieve is untrusted input the next time it enters context. If an agent can read external content and also act, you have to treat retrieved memory as a possible injection vector, which is the lethal trifecta problem ↗ in a different outfit. Compaction and memory are reliability tools, but they widen the surface you have to defend.

What does the full compaction loop look like end to end?

The complete loop is: run the step, check triggers, compact the summary zone against the fixed anchor if a trigger fires, persist durable state to external memory, feed back the context status, and continue. Wired together it looks like this:

def agent_step(state):
    result = run_next_action(state)          # model + tool call
    state.record(result)

    if should_compact(state):
        durable = extract_durable_facts(result, state)
        memory.upsert(durable)               # push state out
        state.summary = compact_summary(     # anchored, iterative
            anchor=state.anchor,             # never rewritten
            prior_summary=state.summary,
            active=state.active_turns,
        )
        state.active_turns = state.active_turns[-RECENT_N:]

    state.append_status_note(context_budget(state))  # feedback
    return state

A few hard-won notes on running this in production. Keep RECENT_N generous; cutting the active zone too aggressively starves the model of the fidelity it needs for the immediate next move. Log every compaction with before-and-after token counts and a hash of the anchor, so when an agent goes off the rails you can replay exactly what it could see. And test compaction explicitly in your evals by running trajectories long enough to trigger it, because a compaction step that silently drops a key identifier is the kind of bug that only surfaces at turn 40.

If you are designing or rescuing a long-running agent and this is the layer that keeps biting you, this is exactly the work I do under AI agents and automation ↗. Most reliability problems I get called in for are not model problems. They are context problems wearing a model problem's clothes.

The short version

Context rot is real, it is measured across every major model, and it gets your agent long before the window does. You beat it by treating context as working memory: anchor the goal so summarization cannot drift, compact on triggers instead of waiting for overflow, tell the model how much budget it has so it cooperates, and offload durable state to external memory so the live context stays lean. Do those four things and an agent that used to collapse at turn 30 will run clean past 100k tokens.

If you are wrestling with an agent that degrades the longer it runs and want a second set of eyes, my contact page ↗ is the fastest way to reach me. I would rather help you fix the context layer than watch another agent quietly rot.