A chat costs what you type plus what the model writes. An agent costs far more, because it calls the model dozens of times and resends almost everything each time. If you want to cut that cost, or just stop hitting the context window, you need to know which part of each request is big. This post breaks an agent loop into five buckets, shows how to measure each from the API’s own usage data, and lists what to trim first.
Why does an agent resend everything?
Chat-completions APIs are stateless. The model remembers nothing between calls, so the agent sends the full context every step. A 40-step task where each call carries 30,000 tokens reads 1.2 million input tokens to produce maybe 40,000 output tokens. That 30-to-1 ratio is why input tokens and their cache price dominate agent bills.
Where do the tokens go?
| Bucket | What it is | Repeats every call? | Typical driver |
|---|---|---|---|
| System prompt | Agent instructions, rules files, project notes | Yes | Long rules files |
| Tool schemas | Name, description and JSON Schema of every tool | Yes | Built-in tools plus every MCP server |
| History | Earlier messages, tool calls and results | Yes, and grows | Long sessions |
| File reads and tool output | File contents, test logs, search results | Yes, until dropped | Reading whole files, verbose commands |
| Output | Visible answer, tool calls and reasoning | No | Reasoning effort, rewriting whole files |
The first two are fixed overhead. Cursor wrote in September 2026 that the system prompt and tool definitions “had become one of the largest sources of spend that we fully control”, and called file reads “another large source of token spend”. Its changes, including cutting about two-thirds of the system prompt and loading tools used in fewer than 20% of conversations only when needed, cut users’ token costs by 7%. GitHub’s VS Code team named the same two costs for Copilot: the repeated prompt prefix and tool-definition overhead.
How do you measure it?
You don’t need a tracing product. The usage block on each response is enough.
- Turn on usage. For streaming, add
stream_options: {"include_usage": true}; usage arrives in the last chunk. See streaming. - Log per call: step number,
prompt_tokens,prompt_tokens_details.cached_tokens,completion_tokensand, when reported,completion_tokens_details.reasoning_tokens. - Measure the fixed overhead once. Send one request with only the system prompt and tools, and the word “hi”. Its
prompt_tokensis your floor. - Measure a tool’s output. Count characters of each tool result before you append it. Dividing by four gives a rough English token count; paste a sample into the free token counter for the exact figure on your model’s tokenizer.
r = client.chat.completions.create(model=MODEL, messages=msgs, tools=tools)
u = r.usage
cached = getattr(u.prompt_tokens_details, "cached_tokens", 0) or 0
log.info("step=%d prompt=%d cached=%d out=%d", step, u.prompt_tokens, cached, u.completion_tokens)
Plot prompt_tokens by step. A steady slope means history is growing. A sudden jump means a large file or log entered the context. A high floor means too many tools.
In Aider, /tokens reports what’s in the current context, and /drop removes files from it.
What should you trim first?
Order by size times repetitions, not by what’s easiest:
- Unused tools. Every tool schema rides on every call. OpenAI’s guide says function definitions “are billed as input tokens” and suggests fewer than 20 functions at the start of a turn, as a soft rule. Disable MCP servers you don’t use in this task; opencode’s docs say plainly that MCP tools consume model context.
- Tool output. Cap command output (last 200 lines of a test log, not 20,000). Return search results as paths and line numbers, then read only what’s needed.
- File reads. Read the function, not the file. Aider’s repo map uses a default 1k-token budget (
--map-tokens) instead of whole files. - History. Start a new task per change. When a session must continue, replace old turns with a short summary.
- System prompt. Remove rules the model already follows. Keep the rest stable.
- Output. Ask for diffs, not whole files. Set
reasoning_efforttolowfor routine steps. Reasoning tokens are billed as output.
Why keep the prefix stable?
Providers cache a repeated prefix and bill it at a lower rate. On BroadNet Inference, DeepSeek V4.1 Flash bills cached input at $0.0055 per 1M tokens and new input at $0.175. GLM 5.3 bills $0.173 against $0.78. A timestamp in the system prompt, or tools listed in a different order each call, breaks the cache from that point on. Put stable parts first, in a fixed order, and changing parts last. More in cached input tokens explained; the prompt caching calculator shows what a given hit rate saves.
Caching lowers the price of the prefix, but not its size. Cached tokens still fill the context window and still take time to process.
What does a worked agent budget look like?
Take the 40-step task above, with 80% of input cached:
| Part | Tokens | Billed at |
|---|---|---|
| New input | 240,000 | Input price |
| Cached input | 960,000 | Cached input price |
| Output incl. reasoning | 40,000 | Output price |
Cutting 5,000 tokens of unused tool schemas from each call removes 200,000 input tokens from this task, about a sixth of the total, with no change to the agent’s behaviour. For the cost levers beyond trimming, such as model choice, caching and flat plans, see 10 ways to cut coding agent LLM costs. For multi-agent setups, multiply by the number of agents; see what multi-agent setups cost. To model your own loop step by step, use the agent run cost estimator.
Where BroadNet Inference fits
BroadNet Inference returns prompt and completion tokens in the usage block, plus cached and reasoning tokens when the model reports them, so the measurement above works as written. Its low cached-input price on DeepSeek V4.1 Flash rewards a stable prefix. If trimming isn’t worth your time, the unlimited plans remove the token meter for DeepSeek V4.1 Flash at $60 a month per session; the context window still applies, and a lean context still runs faster. Plans are on inference.broadnet.ai/plans.
