Context Management

The model has a finite context window, and even inside that window, attention degrades as noise grows. Context management is the harness taking responsibility for that scarcity: deciding, at every single turn, what enters the model right now.

It is the least visible primitive on this sheet. When it works, you experience a sharp agent. When it fails, you experience a model that got worse overnight — and you go off tuning the wrong thing.

An agent that got dumb at turn 40

A worked example, carried the whole way through, because the ideas stay concrete that way.

An agent is debugging a live-streaming app. A phone sends video over WebRTC, a node server re-encodes it with FFmpeg, and it goes out to YouTube. The ticket says that on a real cellular connection the video goes choppy and low-resolution and the audio comes out close to noise. The ticket links to the server's Docker logs, an earlier session where the video side was already worked, and a design document from the pipeline work.

The agent starts well. It reads the ticket, opens the encoder code, pulls the logs, and forms a theory about the encoder restarting over and over.

Around turn 40 it starts acting strangely. It reopens a file it has already edited and asks what is in it. It proposes a fix it ruled out a few turns earlier. It contradicts a decision made with the user twenty turns back. The first half of the conversation has quietly fallen out of its head.

Four ways long context goes bad

"The prompt got too long" is not precise enough to fix. The fixes differ depending on what actually went wrong. Drew Breunig's breakdown gives the four failure modes names worth borrowing:

• Poisoning — a wrong fact or a bad tool result gets into the context, and then keeps getting referenced as if it were true.
• Distraction — the context grows so long the model over-focuses on it, leaning on what is in the window instead of what it learned in training.
• Confusion — superfluous material is present, and the model uses it to produce a worse answer than it would have without it.
• Clash — the context accumulates facts or tool results that contradict each other, and the internal conflict degrades the reasoning.

The streaming session had at least two of them. Distraction, because the Docker log ran to thousands of lines and ate most of the window. And clash, because the design document described the pipeline as already normalising every frame to a constant 1080p — the fix the work was aiming at — while the deployed code did not do that yet. Document and reality disagreed, and the agent trusted the document.

This is what "protect the model's attention" means concretely: keep those four from happening.

Context window — a budget, not a bucket

FOCUSED

instructions · 12kskills · 8khistory · 20kretrieved · 17ktool results · 13k

How much do you dump in?attention on the task: 64%the one relevant filethe whole repo + six API manuals

The window holds what matters. Attention is cheap.

Not delivery, and not memory

This primitive gets blurred with its two neighbours, so it is worth drawing the boundary before going further.

It is not retrieval. Context delivery asks what knowledge exists that could be pulled in. Context management asks the next question: of everything we could pull in, what actually goes into this prompt on this turn?

It is not memory either. The memory pillar is about what survives between runs. This is narrower and more immediate — it is about this window, right now.

Delivery is what is reachable. Memory is what sticks around. Context management is what makes it onto the desk in front of the model for the current turn.

The four moves

The naive approach is to put everything in. Ticket, linked log, design document, every file opened, every tool result so far. Windows are large now — hundreds of thousands of tokens, and a million at the frontier — so why not?

For a short task it genuinely works. It breaks exactly where the streaming session broke: on a long-horizon task, where tool output piles up turn after turn and the model ends up spending its attention reading thousands of lines of historical notes to answer a question about today's behaviour. You did not make the agent smarter by giving it more. You gave it more to be distracted by.

Lance Martin, at LangChain, frames the alternative as four moves: write, select, compress, isolate. Below they are taken in the order they tend to bite on a real task — select, compress, write, isolate — applied to the session above.

00As found01Select02Compress03Write04Isolate

Window98%

Signalburied under 30%

Turn 40. The window is full and the signal is in the middle of it.

Same model, same weights, same settings — the agent reopens a file it already edited and proposes a fix it ruled out twenty turns ago. Nothing about the model changed. What changed is what is in front of it.

Head

Tail

In the window

• Instructions5%AGENTS.md, repo rules, the system prompt
• The ticket4%choppy video, low resolution, garbled audio
• Design document9%describes every frame normalised to 1080p — the deployed code never did⚑ Clash — the document and the deployed code disagree, and the agent trusts the document
• Docker log40%thousands of lines, most of it normal startup noise⚑ Distraction — the log eats most of the window and the model leans on it
• Encoder code7%where the teardown happens — the thing the turn is actually about
• 40 turns of tool results30%half-read files, superseded diffs, a fix already ruled out
• The question3%why does the encoder keep restarting?

Select — the slice, and where it sits

Instead of dumping everything in, pull the slice this turn needs. When the work is about the encoder restarting, bring in the encoder code and the recent server logs. Do not bring in the score overlay rendering, the YouTube authentication flow, or the parts of the log that are ordinary startup noise.

That sounds obvious until you notice how much of it the harness has to do for you. The agent cannot read the whole repository to decide what is relevant — that would defeat the purpose. So selection itself has to be cheap and approximate: embeddings over files, ranking by recency and relevance, a search index, descriptions it can scan without loading the full body.

Tools are the clearest case. Tool selection starts to break down as the tool set grows — trouble in the tens of tools, real failures around a hundred, as rough approximations rather than thresholds to quote. Past some point you cannot hand the model every tool. Mature harnesses retrieve the relevant tools first, the same way they retrieve relevant documents. If you are building your own, this is the first place to reach for retrieval: do not register every tool on every call, embed their descriptions and pull in the handful this task needs.

Selection also runs across time, not just across files. The ticket linked to an earlier session where the encoder's behaviour was already worked out. That conversation is not in the current window — it happened days ago, in a different session. Whether the agent can get it is entirely a harness decision. Some harnesses keep each session to itself, with no built-in way to load another session's history; the earlier thread sits in a file on disk that the agent cannot reach unless you go and find it. Others store every session in a searchable database and hand the model a session-search tool, so a ticket that points back at last week can pull the thread.

Storing the old session is a memory job. Choosing to pull a slice of it into this turn is this primitive.

Compress — what a good compaction keeps

Selection gets you a good slice. The problem on a long task is that even a good slice keeps growing: every turn adds tool output, and the session was 40 turns and climbing.

So compress. Summarise the running history, trim what is no longer load-bearing, replace thousands of raw log lines with the handful that show the problem. The trick is what you keep. A good compaction is not "make it shorter" — it is make it shorter without losing what the next turn needs. The better harnesses preserve the same handful of things:

• The goal being worked towards
• The constraints and decisions already locked in
• What is done, what is in progress, what is blocked
• The key files and facts in play
• The next step

That is roughly what you would write a teammate before going on holiday: enough that they can pick the work up without rereading the whole thread. For this session, a good compaction at turn 40 reads: the goal is to fix the choppy low-resolution video; the YouTube side is ruled out; the suspect is the encoder tearing down and respawning every time the WebRTC resolution changes; here are the two files; the next step is to stop the resolution collapsing upstream. A few hundred tokens standing in for tens of thousands.

How much say you get over this is itself a harness decision. Some let you move the trigger, and fence off the first few and last several messages so a compaction never touches them — which lines up exactly with lost in the middle, since the head and tail are the high-value seats. Some let you swap in your own compaction prompt, which is worth sitting with: the compaction step stops being a black box and becomes something you design. Others fix the threshold and give you only a one-time note about what to keep.

Write — get it out of the conversation

If compression is lossy, then the fix for the things you cannot afford to lose is to stop keeping them in the conversation at all. Write them down outside the window, and reload them on demand.

For this session that is a scratch file. As the agent confirms things, it writes them to a working note: the suspect is the encoder restart storm; here is the log signature with the resolution drops; here is the file where the teardown happens; here is the fix already ruled out, and why. That file lives on disk. It is not competing for space in the window at all — and when the conversation is compacted, the note survives untouched, because it was never in the conversation to begin with.

Dex Horthy, at HumanLayer, takes this into a full workflow. The agent writes a research file and a plan file before it writes any code. The research file lists the relevant files and line numbers, so the agent is not searching blindly later. The plan file spells out the intended changes — and you review the plan, not a diff. Then each implementation step runs in a fresh window seeded by the plan, rather than dragging the whole chat history along. He reports keeping working context under 40% of the window that way.

The habit this turns into

Research, then plan, then implement — with the first two steps writing to disk and the third starting clean.

The research phase output is a one-page note a human can read, instead of a 40-turn transcript. The plan phase is where you catch a misunderstanding at the cost of one comment instead of a day. The implement phase drags nothing along. If a sub-agent does the research, have it write its findings to a file too — then the next agent reads the file, and the search transcript never enters a window at all.

One boundary worth keeping clean: writing a scratch note for the current task is context management. If that note is meant to outlive the task and inform future runs, it has become memory. Same file on disk — different job, depending on whether you reload it next session.

Isolate — one window is sometimes the wrong unit

Select, compress and write all operate on one window. The fourth move admits that sometimes one window is the wrong unit, and splits the work across several. Give a piece of the task its own context, let it run, bring back only the result.

The huge Docker log is exactly the thing you do not want the main window to read. Hand it to a sub-agent whose whole job is to find the restart pattern and the resolution changes. It reads the entire log in its own context, burns whatever tokens it needs, and returns the answer: 226 restarts, and here is the resolution collapse sequence. The main window never sees the other few thousand lines.

The pattern shows up everywhere once you look for it. A sandbox that executes something and returns the value, not ten thousand lines of stdout. A search sub-agent that returns the answer, not the transcript of how it found it.

What isolation buys you is total capacity — five windows hold more than one. What it costs you is coordination, and the moment you are coordinating windows you have handed off to orchestration.

Two harnesses, opposite bets

Real harnesses make these four moves differently, and the contrast clarifies the design space. Two, as described in the source talk:

Neither is wrong. They are different reliability bets, and it is worth knowing which one your harness is making. Both, underneath, do bounded assembly, keep a stable cached prefix, and use progressive disclosure — a lightweight description first, the heavy body only when it is needed.

The cost that hides in the bill

Most treatments of this primitive stop at what is in the window. There is a second question that costs real money: in what order, and how stable is it?

Modern models cache the prefix of your prompt. If the first several thousand tokens are byte-for-byte identical to last turn, you do not pay full price to process them again — the discount can reach 90%. But the moment you change something early in the context, every token after it is uncached and charged in full.

That changes how you read the four moves. Every time you reselect, re-rank, or compress, you are potentially invalidating the cache. A harness that reshuffles its window every turn can be correct and still be slow and expensive. That is the cost that hides in the bill rather than in the output, which is what makes it easy to miss.

Look at what actually reaches the model

All of the above assumes you can see what is in the window. Most of the time you cannot, and that is the quiet reason these problems persist.

When the agent went wonky at turn 40, the first instinct was to rewrite the prompt or change the model. The thing that actually helped was looking at the context: what is in this window right now, and how many tokens is each part eating? The first time you look, it is usually a surprise — half the window is a log you forgot was there, or three copies of the same note, or a tool result from twenty turns ago with no business still being present.

This is where this primitive leans on verification and observability. You cannot tune selection, compression or writing if you are doing it blind.

The question to ask instead

When an agent disappoints you on a long task, do not jump to the model. Walk the four moves and ask which one let you down.

• Select — did it ever have the right thing, or was the right thing just buried?
• Compress — did the summary throw away a detail the next turn needed?
• Write — did it lose work it should have parked on disk?
• Isolate — did one giant context drown a sub-task that deserved its own window?

And under all four, the meta question: could you even see what was in the window when it happened? If the answer is no, start there, because everything else is guesswork.

In this session it was select and compress together. The Docker log took most of the context and buried the signal; then a hasty compaction deleted the signal entirely. The fix was the kind you can actually make — send the log to a sub-agent and have it write the findings to a file, so neither one can crowd the window or be summarised away. Once the window was clean, the agent found the real issue upstream of the encoder. Same model, finally looking at the right few hundred tokens.

A well-fed model is still just a talker

With instructions, delivery, and management in place, the agent is genuinely sharper inside the conversation. It keeps the important parts, retrieves the right files at the right time, and doesn't spend half its window on noise.

But it is still only describing work. It can identify the file. It can propose the command. It can say "the next step is to run the failing test." Great — someone still has to run it. If the work requires action, the model needs a way to ask the outside world to do something.