BroadRouter
Get started

Cached input tokens: how to cut an LLM API bill

By the BroadNet Inference team · 5 min read · Published · Updated

Cached input tokens are prompt tokens the provider has already processed and can reuse, so they are billed at a lower rate. On BroadNet Inference, DeepSeek V4.1 Flash bills cached input at $0.0055 per 1M tokens against $0.175 for new input. Keep the stable part of every prompt first and identical between calls to get the discount.

Most of what an agent sends to an LLM is the same on every turn: the system prompt, the tool definitions, the files it is working on and the conversation so far. Only the last message is new. Prompt caching means the provider does not bill that repeated part at full price.

What is a cached input token?

Every request is made of input and output tokens. Before a model can answer, it processes the whole prompt. If an earlier request started with exactly the same tokens, the serving system can reuse that work instead of doing it again. The reused part is reported as cached input tokens and billed at a lower rate.

Two points follow from how this works:

  • Only the prefix is cached. The match runs from the first token forward and stops at the first difference.
  • The answer is never cached. The model still generates new output, billed at the full output price.

How much do cached tokens cost on BroadNet Inference?

Per 1 million tokens, from the published price file:

Model New input Cached input Output
DeepSeek V4.1 Flash $0.175 $0.0055 $0.52
GLM 5.3 $0.78 $0.173 $3.46
Qwen3.8 27B $0.175 $0.0435 $2.17

For DeepSeek V4.1 Flash the gap is large: compare $0.0055 per 1M cached tokens with $0.175 per 1M new ones. For an agent that resends a 200,000-token repository on each turn, the cache rate decides most of the bill.

How much can caching save? A worked example

A coding agent sends a 200,000-token prompt 100 times in a working day. That is 20 million input tokens a day.

Cache rate New input tokens Cached input tokens
0% 20M at $0.175 per 1M 0
50% 10M at $0.175 per 1M 10M at $0.0055 per 1M
90% 2M at $0.175 per 1M 18M at $0.0055 per 1M

The formula for the daily input bill is new tokens (M) × input price + cached tokens (M) × cached price. At 90%, 18M of the 20M input tokens are billed at the cached price instead of the full input price, so the wider the gap between the two prices, the more of the input bill you save. Output is unchanged. Try your own numbers in the prompt caching calculator.

How do you see cached tokens in the API response?

BroadNet’s API reference says that when the model reports them, the usage object carries prompt_tokens_details.cached_tokens: the prompt tokens read from cache, billed at the cached price. In a stream, ask for usage in the last chunk:

from openai import OpenAI
import os

client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])
messages = [
    {"role": "system", "content": "You are a code reviewer."},  # stable prefix: keep it identical
    {"role": "user", "content": "Review the diff below."},
]
stream = client.chat.completions.create(
    model="broadnet/deepseek-v4.1-flash",
    messages=messages,
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.usage:
        u = chunk.usage
        cached = (u.prompt_tokens_details.cached_tokens if u.prompt_tokens_details else 0) or 0
        print(f"prompt={u.prompt_tokens} cached={cached} rate={cached / max(1, u.prompt_tokens):.0%}")

Log that rate per request for a day. It tells you what you really pay, and it is the number to put in the prompt caching calculator. More on streaming usage in the streaming guide.

How does prompt caching differ between OpenAI, Anthropic and DeepSeek?

Caching is not standardised. What the official docs said when we fetched them on 14 November 2026:

OpenAI Anthropic DeepSeek (direct API)
Turned on By default for supported models Opt in with a cache_control field (automatic or per block) By default for all users
Minimum prompt 1,024 tokens on GPT-5.6 and later 512 to 4,096 tokens, by model No minimum stated
Cache lifetime 30 minutes by default on GPT-5.6 and later; a 12-hour option 5 minutes by default, 1 hour at extra cost Best effort; cleared when unused, “a few hours to a few days”
Writes 1.25× input (30 minutes) or 2.5× (12 hours) on GPT-5.6 and later 1.25× input for 5 minutes, 2× for 1 hour No write charge mentioned
Reads 0.1× input on GPT-5.6 and later (0.05× on GPT-6.1 Sol and Terra) 0.1× input on most models, lower on some Separate “cache hit” price
Usage field cached_tokens in the token details cache_read_input_tokens prompt_cache_hit_tokens

Sources: OpenAI prompt caching guide, Anthropic prompt caching docs and DeepSeek context caching docs. Where writes cost extra, a cache that is written but never read costs more than no cache at all. Caching pays off only on prompts you actually repeat. To compare list prices per model, see the LLM API pricing table.

What lowers your cache rate?

Caches match an exact prefix, so anything that changes early in the prompt makes the rest of it new. The usual causes:

  • a timestamp, request ID or user name near the top of the system prompt;
  • tool definitions or JSON serialised in a different key order on each call, which changes the tokens;
  • an agent that rewrites or summarises its history instead of appending to it;
  • long pauses, after which the cache has expired.

The fix is a fixed layer order (system prompt, tools, documents, history, new message) with each layer byte-identical between calls. Our guide on how to structure prompts so the prefix caches walks through that layout in code, with a test that proves the prefix is reused.

When does caching stop mattering?

On a flat plan the bill does not depend on tokens at all. Full Moon is $60 a month per session for unlimited DeepSeek V4.1 Flash. If your agent already crosses the break-even point, the cache rate stops being a cost problem. On long prompts the habits above still help with speed, so they are worth keeping either way.

Why does caching matter more with long context?

DeepSeek V4.1 Flash and GLM 5.3 accept up to 1,048,576 tokens per request. Caching is what makes very long prompts affordable on a per-token key, because the large, repeated part of the context window is billed at the cached rate. See when a 1M-token context window is worth it.

Where BroadNet Inference fits

BroadNet publishes a cached-input price for each chat model that has one and reports cached tokens in usage when the model reports them, so you can measure your rate before you commit. It does not publish a cache lifetime or minimum prompt length, so measure those on your own traffic. If your agent runs all day, a flat plan removes the question entirely.

Questions

Do I need to turn prompt caching on?

It depends on the provider. OpenAI and DeepSeek cache automatically; Anthropic needs a cache_control field. On BroadNet Inference, check the usage block: when the model reports cached tokens, they appear as prompt_tokens_details.cached_tokens and are billed at the cached input price.

How do I check if prompt caching is working?

Ask for usage on every request and compute cached_tokens divided by prompt_tokens. A rate near zero on the second call of a long, repeated prompt means something early in the prompt is changing between calls.

Why is my cache rate low?

Usually because something early in the prompt changes on every call, such as a timestamp, a request ID or a reordered tool list. Move changing content to the end.

What is the difference between prompt caching and response caching?

Prompt caching reuses the provider's work on a repeated prompt prefix, and the model still writes a fresh answer. Response caching is something you build yourself, storing whole answers and returning them for identical questions without calling the model.

Are cached tokens free?

No. They are billed at a lower rate, not at zero. On DeepSeek V4.1 Flash, cached input is $0.0055 per 1M tokens, and output is billed in full.

Does caching matter on an unlimited plan?

Not for the bill, because plan sessions have no token meter. It still shortens the time to first token on long prompts in many serving systems, so the same habits help.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity