---
title: "What is context engineering? Managing tokens for AI agents"
description: "Context engineering is choosing which tokens an LLM sees on each turn: instructions, tools, history and retrieved data. Agent techniques and their costs."
url: https://broadrouter.com/glossary/context-engineering/
lang: en
updated: 2026-11-15
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Context engineering

Updated 2026-11-15

Context engineering is the work of choosing and maintaining the set of tokens a language model sees on each request, including system prompt, tool definitions, history, retrieved documents and notes, so the model gets the smallest set of high-signal information it needs to do the task well.

Also called: context management, agent context engineering

A prompt is what you write once. Context is everything the model sees on each turn, and in an agent it changes every turn.

## How it works

Anthropic’s engineering post describes context engineering as “the set of strategies for curating and maintaining the optimal set of tokens” during inference, including information beyond the prompt. It sets the goal as finding the “smallest possible set of high-signal tokens” that makes the desired outcome likely.

The reason is limited attention. The post says models have an “attention budget,” and that recall falls as context grows, which it calls [context rot](https://broadrouter.com/glossary/context-rot/). Position matters too: facts in the middle of a long prompt are easier to miss ([lost in the middle](https://broadrouter.com/glossary/lost-in-the-middle/)).

The post names several techniques for long-running agents:

| Technique | What it does |
| --- | --- |
| [Compaction](https://broadrouter.com/glossary/context-compaction/) | Summarize history near the limit and continue from the summary |
| Tool result clearing | Drop old raw tool outputs from the history |
| Structured note-taking | The agent writes notes to a file outside the context and rereads them later |
| [Subagents](https://broadrouter.com/glossary/subagent/) | Do focused work in a clean context and return a short summary |
| Just-in-time retrieval | Keep references (file paths, queries) and load the data only when needed |

A worked example of one turn’s budget in a coding agent:

```text
System prompt + rules          4,000 tokens
Tool definitions               6,000 tokens
Notes file                     1,500 tokens
History (compacted)           20,000 tokens
Latest tool output             8,000 tokens   <- largest single item
```

## Why it matters

-   **Quality.** Less noise in the context usually means better answers and fewer missed instructions.
-   **Cost.** On per-token billing, every token in the context is billed on every turn. Stable content placed first is cheaper as [cached input](https://broadrouter.com/glossary/cached-input-tokens/); see [prompt structure for caching](https://broadrouter.com/blog/prompt-structure-for-caching/).
-   **Length of run.** Good context management lets an agent work for hours without hitting the [context window](https://broadrouter.com/glossary/context-window/).

## On BroadNet Inference

Context engineering happens in your code: BroadNet’s Chat Completions API receives whatever message list you send. [DeepSeek V4.1 Flash](https://broadrouter.com/models/deepseek-v4.1-flash/) and [GLM 5.3](https://broadrouter.com/models/glm-5.3/) accept 1,048,576 tokens, [Qwen3.8 27B](https://broadrouter.com/models/qwen3.8-27b/) 262,144 and [Gemma 4 VL 26B](https://broadrouter.com/models/gemma-4-vl-26b/) 32K, so a tight context matters most on Gemma.

When the model reports them, `usage.prompt_tokens_details.cached_tokens` shows the tokens read from cache, which are billed at the cached input price on pay-as-you-go. [Context management for long-running agents](https://broadrouter.com/blog/long-running-agent-context-management/) gives five working rules, and the [agent token budget](https://broadrouter.com/blog/agent-token-budget/) post shows where agent tokens go.

## Questions

### How is context engineering different from prompt engineering?

Prompt engineering is about writing good instructions. Context engineering covers everything the model sees on a turn, including tool definitions, history and retrieved data, and how that set changes over a long session.

### Why not just use a bigger context window?

A bigger window delays the problem but does not remove it. Anthropic notes that recall drops as context grows, and on per-token billing every extra token is paid again on each turn.

### What is the first thing to try?

Measure what fills your context. Tool output and old history are usually the largest parts, and trimming or summarizing them is cheap to try.

## See also

-   [Prompt engineering](https://broadrouter.com/glossary/prompt-engineering/) (term)
-   [Context compaction](https://broadrouter.com/glossary/context-compaction/) (term)
-   [Context rot](https://broadrouter.com/glossary/context-rot/) (term)
-   [Lost in the middle](https://broadrouter.com/glossary/lost-in-the-middle/) (term)
-   [Context window](https://broadrouter.com/glossary/context-window/) (term)
-   [Cached input tokens](https://broadrouter.com/glossary/cached-input-tokens/) (term)
-   [How the unlimited LLM API plans work for agents](https://broadrouter.com/guides/unlimited-llm-api-sessions/) (guide)
-   [Context management for long-running agents with a 1M window](https://broadrouter.com/blog/long-running-agent-context-management/) (article)
-   [Agent token budget: where an agent loop's tokens go](https://broadrouter.com/blog/agent-token-budget/) (article)
-   [How to structure prompts so the prefix caches](https://broadrouter.com/blog/prompt-structure-for-caching/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-context-engineering&utm_content=page-cta) ·

**Sources**

-   [Anthropic Engineering, Effective context engineering for AI agents (checked 2026-11-15)](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents)
-   [Claude Code docs, subagents (preserve context) (checked 2026-11-15)](https://code.claude.com/docs/en/sub-agents)
-   [Claude Code docs, manage costs effectively (reduce token usage) (checked 2026-11-15)](https://code.claude.com/docs/en/costs)
