Most of what an agent sends to an LLM is the same on every turn: the system prompt, the tool definitions, the files it is working on and the conversation so far. Only the last message is new. Prompt caching means the provider does not bill that repeated part at full price.
What is a cached input token?
Every request is made of input and output tokens. Before a model can answer, it processes the whole prompt. If an earlier request started with exactly the same tokens, the serving system can reuse that work instead of doing it again. The reused part is reported as cached input tokens and billed at a lower rate.
Two points follow from how this works:
- Only the prefix is cached. The match runs from the first token forward and stops at the first difference.
- The answer is never cached. The model still generates new output, billed at the full output price.
How much do cached tokens cost on BroadNet Inference?
Per 1 million tokens, from the published price file:
| Model | New input | Cached input | Output |
|---|---|---|---|
| DeepSeek V4.1 Flash | $0.175 | $0.0055 | $0.52 |
| GLM 5.3 | $0.78 | $0.173 | $3.46 |
| Qwen3.8 27B | $0.175 | $0.0435 | $2.17 |
For DeepSeek V4.1 Flash the gap is large: compare $0.0055 per 1M cached tokens with $0.175 per 1M new ones. For an agent that resends a 200,000-token repository on each turn, the cache rate decides most of the bill.
How much can caching save? A worked example
A coding agent sends a 200,000-token prompt 100 times in a working day. That is 20 million input tokens a day.
| Cache rate | New input tokens | Cached input tokens |
|---|---|---|
| 0% | 20M at $0.175 per 1M | 0 |
| 50% | 10M at $0.175 per 1M | 10M at $0.0055 per 1M |
| 90% | 2M at $0.175 per 1M | 18M at $0.0055 per 1M |
The formula for the daily input bill is new tokens (M) × input price + cached tokens (M) × cached price. At 90%, 18M of the 20M input tokens are billed at the cached price instead of the full input price, so the wider the gap between the two prices, the more of the input bill you save. Output is unchanged. Try your own numbers in the prompt caching calculator.
How do you see cached tokens in the API response?
BroadNet’s API reference says that when the model reports them, the usage object carries prompt_tokens_details.cached_tokens: the prompt tokens read from cache, billed at the cached price. In a stream, ask for usage in the last chunk:
from openai import OpenAI
import os
client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])
messages = [
{"role": "system", "content": "You are a code reviewer."}, # stable prefix: keep it identical
{"role": "user", "content": "Review the diff below."},
]
stream = client.chat.completions.create(
model="broadnet/deepseek-v4.1-flash",
messages=messages,
stream=True,
stream_options={"include_usage": True},
)
for chunk in stream:
if chunk.usage:
u = chunk.usage
cached = (u.prompt_tokens_details.cached_tokens if u.prompt_tokens_details else 0) or 0
print(f"prompt={u.prompt_tokens} cached={cached} rate={cached / max(1, u.prompt_tokens):.0%}")
Log that rate per request for a day. It tells you what you really pay, and it is the number to put in the prompt caching calculator. More on streaming usage in the streaming guide.
How does prompt caching differ between OpenAI, Anthropic and DeepSeek?
Caching is not standardised. What the official docs said when we fetched them on 14 November 2026:
| OpenAI | Anthropic | DeepSeek (direct API) | |
|---|---|---|---|
| Turned on | By default for supported models | Opt in with a cache_control field (automatic or per block) |
By default for all users |
| Minimum prompt | 1,024 tokens on GPT-5.6 and later | 512 to 4,096 tokens, by model | No minimum stated |
| Cache lifetime | 30 minutes by default on GPT-5.6 and later; a 12-hour option | 5 minutes by default, 1 hour at extra cost | Best effort; cleared when unused, “a few hours to a few days” |
| Writes | 1.25× input (30 minutes) or 2.5× (12 hours) on GPT-5.6 and later | 1.25× input for 5 minutes, 2× for 1 hour | No write charge mentioned |
| Reads | 0.1× input on GPT-5.6 and later (0.05× on GPT-6.1 Sol and Terra) | 0.1× input on most models, lower on some | Separate “cache hit” price |
| Usage field | cached_tokens in the token details |
cache_read_input_tokens |
prompt_cache_hit_tokens |
Sources: OpenAI prompt caching guide, Anthropic prompt caching docs and DeepSeek context caching docs. Where writes cost extra, a cache that is written but never read costs more than no cache at all. Caching pays off only on prompts you actually repeat. To compare list prices per model, see the LLM API pricing table.
What lowers your cache rate?
Caches match an exact prefix, so anything that changes early in the prompt makes the rest of it new. The usual causes:
- a timestamp, request ID or user name near the top of the system prompt;
- tool definitions or JSON serialised in a different key order on each call, which changes the tokens;
- an agent that rewrites or summarises its history instead of appending to it;
- long pauses, after which the cache has expired.
The fix is a fixed layer order (system prompt, tools, documents, history, new message) with each layer byte-identical between calls. Our guide on how to structure prompts so the prefix caches walks through that layout in code, with a test that proves the prefix is reused.
When does caching stop mattering?
On a flat plan the bill does not depend on tokens at all. Full Moon is $60 a month per session for unlimited DeepSeek V4.1 Flash. If your agent already crosses the break-even point, the cache rate stops being a cost problem. On long prompts the habits above still help with speed, so they are worth keeping either way.
Why does caching matter more with long context?
DeepSeek V4.1 Flash and GLM 5.3 accept up to 1,048,576 tokens per request. Caching is what makes very long prompts affordable on a per-token key, because the large, repeated part of the context window is billed at the cached rate. See when a 1M-token context window is worth it.
Where BroadNet Inference fits
BroadNet publishes a cached-input price for each chat model that has one and reports cached tokens in usage when the model reports them, so you can measure your rate before you commit. It does not publish a cache lifetime or minimum prompt length, so measure those on your own traffic. If your agent runs all day, a flat plan removes the question entirely.
