Context & Memory

An agent's context window is its working memory: one finite token budget that every message — user turns, model replies, tool results — must share. Context compaction is how a harness keeps long sessions alive: near the limit, old messages are replaced by a model-written summary, while durable facts go to a memory file that is re-read at every session start.

The picture to keep is a desk and a filing cabinet. Everything the model can currently see is on the desk, and the desk is small. When it overflows, the harness sweeps the old piles into a single sticky note — legible, useful, and missing most of what was swept. The filing cabinet is the memory file: nothing gets there unless someone deliberately walks it over, and in this fixture it is the only information store restored for the next session. Real harnesses may also persist transcripts or summaries. This lesson runs one coding session into the desk's limit three times, under three policies, so you can watch exactly what each mechanism saves and what it quietly costs.

How Context Compaction Works

Start with why windows fill at all. Prose is cheap — a user request is a few hundred tokens. Tool results are not: in the animation's session, one read("src/parser.ts") returns a 19,000-token result that is taller than the entire conversation before it, and a test run dumps 22,100 more. Large tool outputs dominate this example; user messages, model outputs, and other supplied context can also fill a window. The block heights in the animation are drawn to scale (small messages are floored to a readable minimum) precisely so you feel this — the meter beside the column carries the exact arithmetic.

The harness defends the window with a gate it checks before every model call: if the context exceeds 92% of the limit, compact first. Compaction is itself an LLM call — the old messages (everything between the standing header and the last couple of messages) are handed to a model with an instruction like "summarize: goal, decisions, open work," and the context is rebuilt as [system prompt, memory file, summary, recent tail]. The recent messages survive verbatim; everything older now exists only as the summary describes it. The gate sits at 92% rather than 100% because the model still needs headroom to answer — a request that arrives already at the limit is rejected, not summarized.

Memory is the separate, smaller, sturdier mechanism. When the model decides a fact is durable — in this session, the user's "always pnpm, never npm" — it writes the fact to a file on disk. That write does not change the current context at all: the in-context memory block was injected when the session started and is a snapshot, not a live view. The payoff comes later. A summary is a context message, and context dies with the session; the file is read back in at the top of every new session, which makes it the only channel that crosses the boundary.

The three recipe buttons under the Inputs panel isolate these two mechanisms (the threshold and memory tabs set each axis directly). The default runs both. No compaction removes the gate and rides the same conversation into the wall. No memory compacts but persists nothing — and fails in a way that takes a full day to show up.

Context & Memory Step by Step

The default run, with the toy 50,000-token window the animation uses (so the threshold is 46,000). Every number below comes from the trace.

1. Session 1 boots non-empty. The system prompt (3.0k) and the injected MEMORY.md copy (1.0k) land before anyone says anything: 4.0k used.
2. The task arrives. "Fix the failing parser test. Always pnpm, never npm." (0.5k) — a task and a preference in one breath. The model replies that it will note the rule and read the parser (0.8k).
3. The memory write. A fact chip flies from the reply to the file: MEMORY.md += "Always pnpm, never npm". Watch the meter — it does not move. This modeled write does not refresh the injected memory block, so the displayed count does not change. A real memory-writing tool request or result can itself consume context.
4. The window fills. read("src/parser.ts") (0.3k) returns 19.0k of file — the bug the agent is chasing (an off-by-one loop bound at line 247) sitting beside incidental notes it is not: a MAX_DEPTH recursion guard, a TODO about unicode surrogate pairs. The model names the bug and the fix (0.9k), runs bash("apply patch && pnpm test") (0.3k), and the test log lands at 22.1k. Total: 47.9k of 50k.
5. The gate trips. 47.9k exceeds 46k, so the harness compacts before the next model call — the reply you'd expect next is literally waiting on this.
6. The slice is selected. A bracket appears beside the five messages between the standing header and the recent tail: these, and only these, are about to become one summary. The head (system prompt, memory copy) and the last two messages are not in the bracket.
7. Summarize. A real model (claude-opus-5-5, recorded — not a hand-written line) reads the bracketed messages and writes: "User asked to fix the failing parser test and to always use pnpm, never npm; the bug is an off-by-one loop bound at src/parser.ts line 247 (i <= n should be i < n), and the next steps are to patch it and then run the tests with pnpm." (0.8k). Notice what it kept and what it dropped. The bug, the fix, the pnpm rule — all there. The MAX_DEPTH guard and the unicode TODO that shared that file read — gone from this summary. It distilled the conclusion and left out the raw evidence.
8. Compact. The five bracketed blocks converge into the summary's slot and vanish; the last two messages stay verbatim. The meter drops 47.9k → 27.2k in one motion. The context is now [system, memory, summary, test command, test log].
9. The lossy probe. The user asks: "Wasn't there a TODO about unicode in that file?" The sweep searches the summary and finds nothing — the note was in the 19.0k file read, and the summary kept only the bug it was solving. The model answers honestly: it would have to re-read the file. The detail was on screen a few beats ago, and it is gone — compaction working exactly as designed can lose whatever you didn't distill.
10. Session boundary. The session ends and the whole column dies — summary included. The memory panel is the only thing on stage that doesn't flinch.
11. Session 2 boots from the file. System prompt plus a re-read MEMORY.md — now carrying the pnpm line (1.1k): 4.1k used, no history at all.
12. The survival probe. "New repo — install date-fns for me." The sweep finds "Always pnpm, never npm" in the memory block and the model answers pnpm add date-fns. Yesterday's correction, intact, from a file the model never had to remember.

Switch to no compaction and the same conversation keeps climbing: 47.9k, 48.5k, 49.0k — then a routine 4.2k changelog read lands the context at 53.5k, and the next API call is rejected with context_length_exceeded. No degradation curve, no warning shot: the request never reaches a model, so there is nothing the model can do about it. Switch to no memory and compaction performs honorably — the summary even carries "always use pnpm, never npm," so the in-session install probe succeeds. Then the session ends, session 2 boots with a bare system prompt, and the agent types npm install lodash: the exact mistake the user corrected yesterday, repeated because nothing carried the correction across the night.

What survives what?

The lesson's real content is a hierarchy of lifetimes. An old message lives until the next compaction; the last two stay verbatim. A summary lives until the session ends — unless a later compaction summarizes it again. A memory-file line lives until someone deletes it. Every piece of information an agent handles is implicitly assigned to one of these tiers, and most of the assignment happens by default rather than by decision — which is exactly how a stray TODO can evaporate while the bug you were chasing survives.

Be precise about what a summary keeps, because the split is not what you'd guess. It is tempting to say coordinates die: that line numbers and exact values are the first casualties. They are not. The summaries in the animation are recorded from a real model, and in every recorded run that covered the diagnosis — five per summary card — the summary kept parser.ts line 247, the i < n fix and the pnpm rule, because those are the conclusion, the whole reason the session existed. The raw evidence it was never asked about is a different story: the single 92% summary dropped the unicode TODO in 4 of 5 runs, and the 50% run's summary of a summary dropped it in only 2 of 5. Same model, same messages, different outcome. That is the real lesson: what a summary keeps beyond the conclusion is not something you can count on in either direction. Anything you will need verbatim after a compaction — especially something you merely noticed but didn't act on — must either be re-derivable (re-read the file) or promoted to the memory tier before the sweep.

The threshold itself is a trade-off handle, and the Inputs panel lets you hold it: a gate at 92% compacts late — one sweep, but it rides within a few thousand tokens of the wall first; drop it to 50% with MEMORY.md loaded and the gate trips twice on this very task, and the second pass sweeps up the first summary along with everything else — a summary of a summary, with another opportunity to omit or distort details. Compact early and you pay the summarizer often and lose details sooner; compact late and one oversized tool result can shove you into the wall before the gate ever fires. Turn the gate off entirely and the run doesn't degrade — it dies at the 50k wall with an API error. There is no setting that loses nothing; there are only different schedules for the loss.

And why not simply buy a bigger desk? Model limits vary. Larger retained inputs still consume window space and can add processing cost or latency, though prefix caching and provider pricing change the relationship. Compaction is not a workaround for small windows — it is the recognition that working memory and durable knowledge are different storage classes. Compaction buys you the session. Memory buys you tomorrow.

Edge Cases

• The gate runs before the call, not after. The check tokens(ctx) > 0.92 × LIMIT happens on append, and keeps the requests within the limit in this fixture’s compacting runs. A production harness must also recheck after compaction and reserve enough output space; an oversized retained tail can still fail. Remove the gate and the overflow surfaces as an API error, not a harness decision.
• The recent tail can be the heavy part. Compaction here keeps the last two messages verbatim — and one of them is the 22.1k test log, so the context only drops to 27.2k, not to 4k. A harness that needs more headroom trims old tool results first.
• The summary is model-written, and inherits model failure modes. If the summarizer mis-states a decision, the error becomes the official history — every later turn conditions on it with full confidence.
• Memory can go stale. "Always pnpm" persists even if the team switches tooling next quarter. Files that outlive sessions also outlive their truth; curation (and deletion) is part of the mechanism, not an afterthought.
• A second compaction compacts the first summary. Set the threshold to 50% with MEMORY.md and watch it happen: the gate trips on the diagnosis, then again on the test log, and the second sweep's slice contains the first summary — a summary of a summary, one more pass that decides what to keep. In the recorded run shown here, that pass kept the unicode TODO, so the 50% probe finds it while the 92% probe does not.

Common Mistakes

• Believing the agent remembers the session. It doesn't. The model is stateless between calls; the context is the memory, and the context dies. In this fixture only the memory file is restored next session; other products may retain full histories or summaries too.
• Expecting raw evidence to survive compaction. The agent will still know the bug it fixed. Whether it knows the unrelated TODO that shared the file read depends on the run — four of five recorded summaries dropped it. If it matters later, write it down or plan to re-derive it — the summary reliably keeps conclusions, not the material under them.
• Treating the memory file as a log. Writing everything to memory rebuilds the original problem one tier up — the injected copy costs context every single session. Memory earns its keep by being small and curated.
• Confusing the in-context memory block with the file. The block is a snapshot taken at session start. A remember() call mid-session updates the file, not the block — the new line arrives with the next session's injection.
• Leaning on the summary across sessions. In the no-memory run the summary held the preference and the agent still failed the next day. A summary is context. Context dies.
• Fixing overflow by raising the limit. A bigger window fills on schedule — the 19k file reads come along regardless. The discipline that scales is deciding what deserves which lifetime, not renting a larger desk.

A Note on Simplification

The 50k window is a teaching choice rather than a model specification. The file is simulated by a separate memory-state array; this lesson does not write MEMORY.md on your disk. The token counts are scripted so the arithmetic stays legible, where a real harness would measure them with a tokenizer. The compaction summaries themselves are not scripted: each is recorded verbatim from a real model (claude-opus-5-5) compacting these exact messages, which is how we know the bug coordinate survives every time while the unicode TODO survives only sometimes. Real systems are also gentler and more layered than one gate. They trim or truncate old tool results before resorting to a full summary, cache the unchanging prefix so re-reading context is cheaper than it looks, expose manual triggers (Claude Code's /compact) alongside the automatic one, and vary the threshold by product. The shape this lesson teaches — a lossy working tier, a model-written summary at a threshold, and a durable file re-injected at session start — is real; the numbers around it are pedagogy.