Information in this post reflects publicly available sources as of October 17, 2026.
You start a coding agent on a refactor and give it one firm rule: do not touch the database migrations. For twenty minutes it is excellent. It reads the right files, makes clean changes, and runs the tests after each one.
Forty minutes in, it edits a migration. It also retries a fix it already tried and abandoned, and it asks you a question you answered at the start. Nothing changed about the model. What changed is everything the model was looking at.
That is the problem context engineering addresses: deciding what goes into the model's context window, in what form, and in what order, at every step. As agents run longer, it has become the skill that separates agents that hold together from agents that drift.
The window is all the model has
The agents post in this series made one point that drives everything here: the model is stateless. On each call it sees only what is in its context window, and it remembers nothing between calls. Your rule about migrations exists for the model only as long as it is in the window and noticeable among everything else.
Windows are now large, from a few hundred thousand tokens to a couple of million depending on the model. It is tempting to treat that as solving the problem. It does not, for two reasons.
First, everything in the window is paid for on every call, as the earlier posts showed with agent loops and MCP tool definitions. Second, models get worse as the window fills.
In July 2025, Chroma published research testing 18 models from four vendors. It found that accuracy dropped as input length grew, even when the task stayed exactly the same, and that plausible but irrelevant passages hurt more than random filler. The effect has a memorable name: context rot.
So the window is not storage you can fill freely. It is closer to a desk. Whatever is on it gets read, and a cluttered desk makes the important page harder to find.
How it works
From prompt engineering to context engineering
Prompt engineering was about wording: how to phrase an instruction so the model follows it. That still matters, but in an agent the instruction you wrote is a small fraction of the window. The rest is tool definitions, retrieved documents, conversation history, and pages of tool output, most of it assembled by code rather than written by a person.
Context engineering treats the whole window as something you design. It asks what the model needs to see for this step, what it does not, and how each piece should be arranged. Wording is one input. Selection, compression, and ordering are the rest.
Anatomy of a context window
A typical agent's window has the same layers on every call. The order is not arbitrary.
Stable material goes first because of prompt caching. Providers can reuse the processed form of a prompt prefix they have seen recently, and charge much less for it, but only if the prefix is byte-for-byte identical. Anything that changes invalidates the cache from that point onward. Volatile material goes last for a related reason: models tend to attend best to the start and end of a long input, so the current request belongs at the end.
Four moves
Almost every context technique is one of four moves. A useful framing, popularized by LangChain's Lance Martin in 2025, calls them write, select, compress, and isolate.
Write moves information out of the window. A coding agent keeps a to-do list and working notes in files, so after a long session it rereads two short files rather than a hundred old tool outputs. Large outputs follow the same pattern: save the 30,000-line log to disk and keep the path plus a short preview in context.
Select brings back only what the current step needs. This is where RAG lives, but the principle is broader. Keep file paths and IDs in context and fetch the content with a tool call when needed. Some hosts apply the same idea to tools themselves, loading a full tool schema only when the model looks for that tool.
Compress shrinks what has already accumulated. The cheapest form is observation masking: replace old tool results with a short placeholder while keeping the agent's own reasoning and actions. The heavier form is compaction, where the model summarizes the session so far and the summary replaces the history. Summaries are powerful but lossy, which matters for the migration rule in our opening story.
Isolate splits the work. A main agent hands a self-contained task, such as "find every caller of this function," to a subagent with a fresh window. The subagent may read fifty files; the main agent receives a ten-line answer. The cost is coordination: subagents cannot see each other's work, so their results can conflict.
Compaction over a long session
Put these together and a long agent session has a recognizable shape. The window grows with each step, and compaction cuts it back before it hits the limit.
The drop in that chart is where sessions go wrong. Everything the model knew that did not make it into the summary is gone. If the migration rule lived only in the third user message, and the summary paraphrased it away, the agent forgot it in the most literal sense. The fix is structural: put durable rules in the system prompt or a memory file, which sit in the stable prefix and survive every compaction.
A context builder
In code, context engineering usually ends up as one function that assembles the window for each call. A simplified version:
def build_context(task, history, notes, retrieved, budget=150_000):
# 1. Stable prefix: identical bytes on every call, so it caches.
# No timestamps, request IDs, or per-call values in here.
prefix = [
{"role": "system", "content": SYSTEM_PROMPT + PROJECT_RULES},
]
# 2. Durable memory: the agent's own notes file, reread each step.
memory = [{"role": "user", "content": f"Your working notes:\n{notes}"}]
# 3. Compress history: keep recent turns verbatim, mask old tool output.
recent = history[-12:]
older = [mask_tool_output(m) for m in history[:-12]]
# 4. Select: only the documents relevant to this step.
docs = [{"role": "user", "content": format_docs(retrieved[:5])}]
# 5. Current request last.
request = [{"role": "user", "content": task}]
messages = prefix + memory + older + recent + docs + request
if count_tokens(messages) > budget:
messages = prefix + memory + [summarize(older)] + recent + docs + request
return messagesThe helpers stand in for your own implementations, but the structure is the point. Rules live in the prefix, where compaction cannot touch them. Old tool output is masked before anything is summarized, because masking is cheap and lossless for the agent's own decisions. And the budget check makes compaction a deliberate step with a known trigger, rather than something that happens when the API returns an error.
When to invest, and when not to
Some context engineering applies to every application. Put stable content first, keep per-call values out of the prefix, and send only the tools a task needs. These cost nothing and pay off immediately in cache hits and accuracy.
The heavier machinery, such as compaction, memory files, and subagents, earns its keep in long-running agents: coding sessions, research tasks, anything past a few dozen steps. A chatbot answering one question per conversation does not need a compaction strategy. It needs a good system prompt and clean retrieval.
A practical signal: if quality drops as sessions get longer, and starting a fresh session fixes it, you have a context problem rather than a model problem. That is the moment to invest.
Start by making the context visible. Log the exact messages sent on each call, not just the user's request, and break the token count down by layer: system prompt, tools, memory, history, tool results. Most teams are surprised by the result. Tool definitions and stale tool output usually outweigh everything a person actually wrote.
Once you can see the window, most fixes are small and local. Trim a verbose tool, move a rule into the system prompt, drop an unused MCP server, or cap a retriever at five results instead of twenty. Each change is easy to measure against the same session replayed before and after.
What goes wrong
Summaries drop the one thing that mattered. Compaction keeps the gist and loses specifics: the exact error message, the approach already ruled out, the constraint the user stated once. Tell the summarizer explicitly what to preserve, such as decisions, rejected approaches, and open questions, and keep hard rules out of the conversation history entirely.
One timestamp breaks the cache. A system prompt that begins with the current time, or a request ID, changes on every call. Every token after it misses the cache, and costs quietly multiply. The same thing happens when tools are listed in a different order each time, which is why the current MCP spec asks servers to return tools in a deterministic order.
Tool output floods the window. One unbounded grep or a full API response can drop 50,000 tokens into context in a single step. Cap tool output at the tool, return counts and previews, and let the model request more.
Stale context contradicts current context. The model read config.yaml at step 3, edited it at step 9, and is now reasoning from the old version still sitting in history. Masking or replacing superseded tool results prevents the model from trusting the wrong copy.
Instructions get buried. A constraint stated once in a long system prompt, between paragraphs of background, competes with everything around it. Keep the system prompt short and specific, put hard rules in a clearly marked list near the top, and move reference material into documents the agent can fetch when it needs them.
More context feels safer and works worse. Adding every possibly relevant document is the most common mistake. Context rot research shows that plausible but irrelevant material, exactly the kind a generous retriever returns, is what hurts most. Fewer, better passages beat more of them.
Cost and latency
Context engineering is where agent costs are won or lost, and caching is the biggest lever. Pricing models differ: Anthropic marks cache points explicitly, while OpenAI and Google cache automatically on recent models. On current Anthropic pricing, for example, cached reads cost one-tenth of the normal input price, and writing to a five-minute cache costs 1.25 times the normal price.
Take an agent with a 30,000-token stable prefix running 20 steps, at an illustrative $3 per million input tokens. Without caching, the prefix alone is sent 20 times: 600,000 tokens, or $1.80. With caching on those Anthropic-style rates, the first step writes the cache at 1.25 times the price and the other 19 read it at a tenth. That comes to about $0.28, a saving of roughly 84% on the prefix.
Latency improves too, because cached prefixes skip most of the processing. A long, stable prefix with a short, changing tail is fast and cheap. A prompt that changes near the top is slow and expensive at any length.
Compaction has its own cost: summarizing is an extra model call, and a lossy summary can cost more steps later as the agent rediscovers what it forgot. Masking old tool output is usually the better first step, because it removes the bulk of the tokens without a model call and without touching the agent's own reasoning.
The current landscape
Context windows on frontier models now range from a few hundred thousand tokens to around two million. Prompt caching is available from all three major providers, with different activation models and pricing, so check each provider's current pricing page before forecasting costs.
Agent harnesses build these techniques in. Claude Code compacts automatically as the window approaches its limit, reads project instructions from CLAUDE.md files, and uses subagents for isolated tasks. Many coding agents also read AGENTS.md, a shared convention for project instructions that is now a project of the Agentic AI Foundation. The OpenAI Agents SDK provides sessions for managing conversation history, and LangGraph lets you define exactly which state fields reach the model at each step.
The protocols in this series feed into this as well. MCP servers add tool definitions to the prefix, so every connected server has a context cost. RAG is the select move done at scale. And the hard problems in multi-agent systems are mostly about isolation: what each agent sees, and what it hands back.
Summary
An agent's behavior at any step is a function of what is in its context window and nothing else. When an agent forgets a rule, repeats a mistake, or drifts off task, the cause is almost always visible in what it was shown. The model did not change between minute one and minute forty. Its view of the world did.
That makes the context window the main design surface for any AI application that runs more than a single call.
Put durable rules where compaction cannot reach them. Keep the stable parts stable so they cache. Retrieve narrowly, mask aggressively, and delegate work that would flood the window. The best agents do not have more context; they have the right context at each step.
Part of the Applied AI series: AI from the builder's side.