BroadRouter
Get started

Context window

Updated

A context window is the maximum number of tokens a language model can work with in a single request. It covers the prompt, conversation history, tool definitions, images and the generated output, including any reasoning tokens. Anything that does not fit must be cut, summarised or split across several requests.

Also called: context length, context size, token limit, context limit

The context window is a model’s working memory for one request. Everything the model needs to see, and everything it writes back, has to fit inside it.

How it works

The window is measured in tokens. Anthropic’s docs describe it as “all the text a language model can reference when generating a response, including the response itself.” That includes:

  • the system prompt and every message, including earlier turns;
  • tool definitions and tool results;
  • documents and images;
  • the output, including hidden reasoning tokens.

So the budget is shared. With a 1,048,576-token window, a 1,040,000-token prompt leaves about 8,000 tokens for reasoning and answer. Set max_tokens with that in mind.

The model does not remember earlier requests. A chat feels continuous only because the client resends the history each turn, so a long conversation fills the window step by step.

Why it matters

  • Capability. A large window lets one request hold a whole codebase, a contract set or hours of transcripts, so the model can connect facts across them.
  • Cost. Every token in the window is billed as input on a per-token key. Caching lowers the price of repeated prefixes, but cached tokens still occupy the window.
  • Quality. More is not always better. Anthropic calls the loss of accuracy in very long prompts “context rot”. Curate what you send.
  • Latency. Long prompts take longer before the first token appears.

On BroadNet Inference

Model Context window
DeepSeek V4.1 Flash 1,048,576 tokens
GLM 5.3 1,048,576 tokens
Qwen3.8 27B 262,144 tokens
BroadNet Gemma 4 VL 26B 32,768 tokens

The window covers prompt and answer together, and BroadNet does not trim your context. DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason before they answer, so leave room: a full window with a small output budget returns an empty reply. See why reasoning models return empty answers.

The unlimited plans include the full 1M-token window on DeepSeek V4.1 Flash. For when that much context beats retrieval, read the 1M-token context window guide.

Common mistakes

  • Treating the window as output length. Output limits are set separately with max_tokens.
  • Resending a full history forever. Summarize or drop old turns before the window fills.

Questions

What happens if my prompt is longer than the context window?

The API rejects the request, usually with a 400 error. Anthropic, for example, returns "prompt is too long". Some chat apps trim old messages for you; most APIs do not.

Is a bigger context window always better?

No. Long prompts cost more and take longer to process, and Anthropic notes that accuracy and recall can degrade as token count grows. Send what the task needs.

Does BroadNet trim my context?

No. BroadNet Inference does not trim your context. If the request does not fit, shorten it yourself.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity