Context and State

Time: ~12 minutes. Tangible win: name the context problem your loop will hit first, and choose one technique from four to address it.

Every step of the loop appends to the conversation. Tool results, model replies, and retrieved documents all compete for the same finite window. Anthropic's context engineering guidance frames this as the central engineering problem for agents: find the smallest set of high-signal tokens that makes the desired behavior likely.

Primary source Effective context engineering for AI agents, Anthropic. Read the sections on the attention budget, just-in-time retrieval, and long-horizon techniques.

Why more context can hurt

Anthropic describes context rot: as the number of tokens grows, the model's ability to recall specific information from that context decreases. Context is therefore a finite resource with diminishing returns. Every token you add spends some of the model's attention budget, so each addition should earn its place.

Upfront or just in time

You can load information before the run, or give the agent references and let it load what it needs. Anthropic calls the second approach just-in-time retrieval. An agent that holds a file path or a stored query, and loads the file only when it needs it, usually spends less context than one that loads everything first. The trade-off is that exploration takes more steps and needs clear guidance so the agent does not wander.

Four techniques for long tasks

TechniqueWhat it doesBest when
Tool result clearingRemoves old raw tool outputs the agent no longer needsTool calls return large payloads
CompactionSummarizes a long history and starts a fresh window from the summaryLong back-and-forth with many decisions
Structured note-takingThe agent writes progress notes outside the window and rereads themIterative work with clear milestones
Sub-agentsA focused agent explores in its own window and returns a short summaryDeep research that would flood the main context
Start with the lightest lever Anthropic's guidance favors the simplest thing that works. Clearing old tool results is the lightest touch. Reach for sub-agents only when the lighter techniques fail a measured test.

Practice

  1. Add a counter to your loop from lesson 0001 that prints the total characters in messages after each step. Run a multi-step task and watch the growth.
  2. Identify the largest tool result in that run. Decide whether the agent needs the full payload later or only a summary.
  3. Implement one technique from the table. Clearing old results is the simplest. Measure the character count again and confirm the task still succeeds.

Retrieval check

Why can adding more tokens to an agent's context sometimes make it worse?

Correct. Anthropic calls this context rot. The attention budget is finite, so each new token competes with the ones that matter.

Not quite. The cause is the finite attention budget. Recall degrades as the window fills, which is why every token should be high-signal.

Which technique keeps a long task's progress outside the context window?

Correct. The agent writes notes to persistent storage and rereads them when needed, so progress survives beyond the window.

Not quite. Only note-taking stores state outside the window. The other options either add tokens or increase the output limit.

Ask the agent: "Show me how to implement tool result clearing in my loop, and how to test that the task still succeeds." Next lesson covers guardrails, the limits and approvals that keep a loop safe.