A prompt is what you write once. Context is everything the model sees on each turn, and in an agent it changes every turn.
How it works
Anthropic’s engineering post describes context engineering as “the set of strategies for curating and maintaining the optimal set of tokens” during inference, including information beyond the prompt. It sets the goal as finding the “smallest possible set of high-signal tokens” that makes the desired outcome likely.
The reason is limited attention. The post says models have an “attention budget,” and that recall falls as context grows, which it calls context rot. Position matters too: facts in the middle of a long prompt are easier to miss (lost in the middle).
The post names several techniques for long-running agents:
| Technique | What it does |
|---|---|
| Compaction | Summarize history near the limit and continue from the summary |
| Tool result clearing | Drop old raw tool outputs from the history |
| Structured note-taking | The agent writes notes to a file outside the context and rereads them later |
| Subagents | Do focused work in a clean context and return a short summary |
| Just-in-time retrieval | Keep references (file paths, queries) and load the data only when needed |
A worked example of one turn’s budget in a coding agent:
System prompt + rules 4,000 tokens
Tool definitions 6,000 tokens
Notes file 1,500 tokens
History (compacted) 20,000 tokens
Latest tool output 8,000 tokens <- largest single item
Why it matters
- Quality. Less noise in the context usually means better answers and fewer missed instructions.
- Cost. On per-token billing, every token in the context is billed on every turn. Stable content placed first is cheaper as cached input; see prompt structure for caching.
- Length of run. Good context management lets an agent work for hours without hitting the context window.
On BroadNet Inference
Context engineering happens in your code: BroadNet’s Chat Completions API receives whatever message list you send. DeepSeek V4.1 Flash and GLM 5.3 accept 1,048,576 tokens, Qwen3.8 27B 262,144 and Gemma 4 VL 26B 32K, so a tight context matters most on Gemma.
When the model reports them, usage.prompt_tokens_details.cached_tokens shows the tokens read from cache, which are billed at the cached input price on pay-as-you-go. Context management for long-running agents gives five working rules, and the agent token budget post shows where agent tokens go.