Skip to specification

Context Assembly

The engineering problem at the heart of memory: retrieve, rank, and assemble slices of every store into the working context. The model only ever works at the desk — and when the task already fits on it, a memory harness buys nothing but cost. Past that threshold the recall policy is the system: ranked beats similar, a perfect retriever still misses the ceiling, and bad memory is expensive twice over.

Here is the key mental model of the whole pillar: memory is context assembly over time. Episodic memory can live in a session database, semantic memory in a fact store, procedural memory in skill files — none of it helps until some slice of it reaches the context window and shapes the model's next step.

The durable stores are the filing cabinet. Working memory is the desk. The model only ever works at the desk.

filing cabinet → context builder → the desk · click to inspect

Left: the cabinet. Middle: the part of the harness that decides what gets pulled forward. Right: the desk. Click any node.

The context builder

Before the model responds, the harness collects material from several places: the current request, the visible conversation, retrieved past sessions, project facts, loaded skills, tool results, open files. It retrieves the candidates, ranks them, and assembles the prompt.

Storing things is the easy part. The hard questions all live in the builder:

  • What should we store in the first place?
  • When should we retrieve it — and how much should we inject?
  • What should outrank what?
  • How do we handle contradictions?
  • When should an old memory stop helping?

The end-to-end run

Watch every memory type fire on the request from earlier — "Pick up issue #9, the old-club-on-profile bug assigned to Buddy. Use the same approval-gated workflow as last time."

WORKING     the current request itself, in the window
EPISODIC    finds the prior issue-9 session: earlier run was blocked,
            no approval comment existed at the time
SEMANTIC    supplies repo layout, allowed folders, Buddy → buddy-b,
            and the approval policy
PROCEDURAL  loads the approval-gated implement workflow

And the response that good memory produces:

"I found issue #9 and the prior automation history. The earlier run was blocked because approval was missing; the later thread now includes a proposal. I'll read the onboarding doc, verify the approval comment, work inside the expected project roots, run focused tests, open a draft PR, and comment back with the result."

In the wild

When you evaluate an agent product's "memory," get specific about which job it actually does. Session search, memory files, and skills line up almost one-to-one with episodic, semantic, and procedural memory. A vector database on its own is only a retrieval mechanism — the architecture around it decides whether the right slice reaches the desk.

Memory is a loop, not a store

Those five questions usually get answered by naming a product: a vector database, a memory file, a session log. That is the wrong shape of answer, and the second talk this sheet stands on says why. Memory is a write–manage–read control loop wrapped around the model — not a database attached to it. Storage is one third of one of those three verbs.

Drawn as a loop, the builder stops being a retrieval step and becomes the part of the harness with a policy. And a policy can be varied, measured, and got wrong.

That experiment splits the loop into three blocks, which map cleanly onto the layers this pillar has already walked through:

01Core

What it holds

Always in front of the agent, every single turn. The traces it must not be allowed to lose sight of, regardless of what else is competing for room.

Where it lands on this pillar

Working memory — the desk itself, and the part of it that is spoken for before the turn begins.

02Recall

What it holds

Pulled forward per turn, by a policy, from everything the run has produced so far. The only block the experiment varied.

Where it lands on this pillar

This sheet. Retrieve, rank, assemble — episodic and semantic material arriving on demand.

03Archival

What it holds

Kept across sessions, outliving the run that wrote it. Read rarely, but the reason anything survives a restart at all.

Where it lands on this pillar

Semantic and procedural memory — and whatever gets through the hygiene of the next sheet.

The agents underneath were chosen for having zero durable memory of their own, so that every remembered thing had to come from the harness. That is what makes the next result mean anything: hold the model fixed, hold the agent fixed, hold the task fixed, and change one block.

When the harness earns nothing

The first task was a literature review. The corpus contained a Nature paper announcing 742,000 promising new materials — a very large claim, later retracted. The retraction is a far smaller needle than the headlines and the citations that repeat the claim approvingly, which makes it exactly the sort of thing a memory system is supposed to save you from.

The harness changed nothing. Same accuracy with recall and without it, at higher cost — because every paper already fit inside the context window. There was nothing for recall to recover that attention could not reach on its own.

TASK ONE · LITERATURE REVIEWthe retractionthe window holds every paperMemory on, memory off — the same accuracy. The harness added cost and nothing else.TASK TWO · XBENCH — A 500-STEP RUN✦ answer written at step 124question asked at step 500 ▾376 STEPS OUT OF REACH0124the last stretch — all the window holdsNo recall policy answers this at any price. The answer left the window 376 steps ago.The gap is the whole argument for a memory harness. Measure it before you build one.
The same rule twice: the extent of the material, the stretch of it the window holds, and where the answer sits. Step numbers are the talk's; the window's width is drawn to read rather than measured.

The second task came from xbench, an established benchmark for long-horizon memory, and it is built to make that distance unavoidable. The answer is written at step 124. The question is asked at step 500. Nothing about a bigger model, a better prompt, or more careful attention closes 376 steps — the material is simply not there to attend to. Past that threshold the recall policy is not a component of the system. It is the system.

The recall ladder

So the recall block was set four ways, from remembering nothing to being told the answer.

Four recall policies

Same model, same agent, same task — only this block changed

  1. 04The oracle

    The harness is handed the correct memory for every loop. Retrieval solved by fiat — the upper bound, not a system anyone could ship.

    The bound that isn’t

    Did not reach the ceiling, and did not beat the ledger below it. The right memory in the window is not the right memory used: the model can still read past it, misread it, or be confused by it.

  2. 03Ranked decisions ledger⚑ Winner

    Every turn writes down what was decided. Recall pulls the highest-priority decisions — not the nearest neighbours, and not the most recent.

    Best measured

    Best of everything tested, and cheaper than the alternatives. Held under ablation, on a second model, and on a second benchmark.

  3. 02Vector recall

    Embed the trace, retrieve by similarity, inject whatever comes back. The default that ships in most memory features.

    Between the two

    Better than remembering nothing. Similarity is not priority — it has no way to know which turn was the one that mattered.

  4. 01No recall

    The window and nothing else. The baseline every memory claim has to clear before it has earned its cost.

    Baseline

    Fails any question whose answer left the window. Cheapest to run, and useless on the far side of the gap.

Off-ladderOne further condition gated the harness — ask the model whether it needs memory this turn, and skip recall when it says no. It lost to simply ranking every turn. Asking a model what it has forgotten is itself a memory question.

Four settings of one block, ordered by how much the policy claims to know. Ranked recall beat the rung above it — and cost less than the rungs below.

Across 68 questions, with multiple seeds each, the ranked decisions ledger won. Not the embedding search, and not the ground truth. Writing down what was decided on every turn, and then recalling by priority rather than by resemblance, beat every other setting — and it held up under ablation, on a second model, and on a second benchmark.

The ledger's advantage is that it stores the right kind of thing. Similarity search retrieves text that looks like the question; a decisions ledger retrieves the turns that changed the state of the work. Those are rarely the same turns, and only one of them is what step 124 actually was.

Why the oracle misses the ceiling

The oracle rung is the one worth sitting with. It hands the harness the correct memory on every loop — retrieval solved, perfectly, by cheating. It still did not reach full marks, and it did not beat the ledger.

The reason is plain once stated: the oracle supplies the right memory, it does not force the model to use it. The right slice can land on the desk and be read past, misread, outweighed by something louder, or simply ignored. Retrieval accuracy is a ceiling on performance, not a substitute for it.

That should recalibrate how much of the memory problem you think retrieval represents. Even a perfect retriever leaves a gap — and everything in that gap is context management's problem, not memory's: where in the window the recalled material sits, what it is competing with, and whether the agent is told why it is there.

Bad memory is expensive

The last finding is the one that changes budgets. Ranked recall was not only more accurate, it was cheaper. That is not the trade-off anyone expects, and it follows directly from what bad recall does: it spends tokens on material that does not help, and then sends the agent in a direction it has to come back from. You pay twice, once for the tokens and once for the detour.

Which gives the heuristic this sheet ends on: treat the recall policy as a first-class metric. Not the store, not the embedding model, not the size of the window — the policy. What kinds of memory do you write? How do you rank them? What does the recall function actually return on the turn that matters?

Remembering everything is its own failure

The builder solves getting memory in. That leaves the opposite problem: a system that remembers everything eventually remembers too much — and the useful stuff drowns. Ranking is only half an answer, because it decides what wins this turn without ever deciding what should stop competing.