max_tokens is the output budget for one reply. It protects you from runaway answers, but set too low it cuts answers off or returns nothing at all.
How it works
The model generates one token at a time. When the count reaches max_tokens, generation stops, even mid-sentence. The response then says why it ended:
finish_reason |
Meaning |
|---|---|
stop |
The model finished on its own |
length |
The max_tokens limit was reached |
tool_calls |
The model is asking to call a tool |
The parameter has several names. OpenAI’s Chat Completions API now marks max_tokens as deprecated in favor of max_completion_tokens, an upper bound that includes both visible output and reasoning tokens. OpenAI’s Responses API uses max_output_tokens. Anthropic’s Messages API uses max_tokens. Many OpenAI-compatible providers accept max_tokens.
The prompt and the output share one context window. If the window is 32,768 tokens and the prompt is 30,000, at most 2,768 tokens can come back, whatever max_tokens says.
Why it matters
- Cost control. On a per-token key,
max_tokensis the most a single reply can cost in output. - Empty answers. On reasoning models, thinking comes out of the same budget. A limit of 100 can be spent entirely on reasoning tokens, so the visible answer is empty and
finish_reasonislength. - Broken JSON. A cut-off reply leaves structured output incomplete and unparseable. Check
finish_reasonbefore you parse.
On BroadNet Inference
DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason before answering, and BroadNet’s docs say the reasoning counts toward max_tokens. Allow 1,000 tokens or more, leave it unset, or lower reasoning_effort:
response = client.chat.completions.create(
model="broadnet/glm-5.3",
max_tokens=2000,
messages=[{"role": "user", "content": "Summarize this contract clause."}],
)
if response.choices[0].finish_reason == "length":
print("Cut off: raise max_tokens or lower reasoning_effort")
Context sizes differ by model: 1,048,576 tokens for DeepSeek V4.1 Flash and GLM 5.3, 262,144 for Qwen3.8 27B and 32,768 for BroadNet Gemma 4 VL 26B. See the model pages and why reasoning models return empty answers.
Common mistakes
- Copying a small limit from a non-reasoning model during a migration. See the migration checklist.
- Treating
lengthas success. Log it and retry with more room.