---
title: "max_tokens for reasoning models - how to budget | BroadRouter"
description: "Why reasoning models return empty replies at low max_tokens, and how to size the budget from measured percentiles, the context window and a retry that escalates."
url: https://broadrouter.com/blog/reasoning-model-max-tokens-budget/
lang: en
updated: 2026-11-15
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Setting max\_tokens for reasoning models: a budgeting method

By the BroadNet Inference team · 6 min read · Published 2026-11-15

On a reasoning model, max\_tokens caps thinking and answer together, so a tight limit can end the reply before any visible text. Measure total completion tokens per task type, set max\_tokens to about the 95th percentile plus 20%, keep prompt plus max\_tokens inside the context window, and retry cut-off replies with a doubled budget up to a hard cap.

## Key takeaways

-   max\_tokens is a ceiling, not a charge. You pay for tokens generated, so a generous ceiling costs nothing on replies that finish early.
-   A tight ceiling is what costs money: a cut-off reply bills its thinking, returns no answer, and has to be sent again.
-   Budget per task type from your own logs: the 95th percentile of completion tokens plus a 20% margin is a sound start.
-   Prompt tokens plus max\_tokens must fit the context window: 262,144 tokens on Qwen3.8 27B, 1,048,576 on GLM 5.3 and DeepSeek V4.1 Flash.
-   Treat finish\_reason "length" as a signal: retry with a doubled budget or lower reasoning\_effort, with a hard cap.

**On this page**

1.  Why does the budget run out before the answer?
2.  Does a bigger max\_tokens cost more?
3.  How do you measure what a task needs?
4.  How does the context window limit the budget?
5.  What should you do when a reply is still cut off?
6.  How does streaming change the picture?
7.  Which parameter name should you send?
8.  Where BroadNet Inference fits

A reasoning model thinks before it answers, and on most OpenAI-compatible APIs both come out of one budget: `max_tokens`. Set it the way you would for a non-reasoning model, a few hundred tokens for a short answer, and some replies come back empty with `finish_reason: "length"`. The quick fix is in [why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/). This post is the method behind a budget you can trust: measure, size from percentiles, respect the context window, and handle the replies that still get cut off.

For what reasoning costs and how to reduce it, see [reasoning tokens: why you pay for more output than you see](https://broadrouter.com/blog/reasoning-tokens-cost/).

## Why does the budget run out before the answer?

The model generates in order: first its reasoning, then the visible answer. Every token of both counts toward `max_tokens`. BroadNet’s docs say this for DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B: “the reasoning counts toward max\_tokens”, and with a small limit “the answer comes back empty”. OpenAI’s Chat Completions reference defines `max_completion_tokens` the same way, as an upper bound “including visible output tokens and reasoning tokens”.

So the question to ask isn’t “how long is my answer?” It is “how long are reasoning and answer together, on this task, at this effort?” That varies more than people expect. OpenAI’s guide says its models may generate “anywhere from a few hundred to tens of thousands of reasoning tokens”, depending on the problem.

## Does a bigger max\_tokens cost more?

No. This is the most common reason budgets are set too tight. [`max_tokens`](https://broadrouter.com/glossary/max-tokens/) is a ceiling. You are billed for the tokens actually generated. A reply that needs 1,800 tokens costs the same with a ceiling of 4,000 or 40,000.

What a tight ceiling does cost:

| Ceiling vs need | What you pay | What you get |
| --- | --- | --- |
| Ceiling above need | Tokens generated | The answer |
| Ceiling below need | The full ceiling, all of it reasoning | Nothing usable, plus a retry |

Take an illustration. Suppose a quarter of your calls need more than 1,000 tokens. With `max_tokens=1000`, each of those calls bills 1,000 output tokens of thinking, returns nothing and is retried. That adds 250 wasted output tokens per call on average across all calls, before the retries themselves, plus the latency of a second request. With the ceiling at 4,000, nothing is wasted. On [Qwen3.8 27B](https://broadrouter.com/models/qwen3.8-27b/), output costs $2.17 per 1M tokens, so waste adds up at volume.

The real risk of a high ceiling is a runaway reply: a model that loops or over-thinks and generates far more than needed. A hard cap bounds that. Pick the cap from data, as below.

## How do you measure what a task needs?

Run real prompts for one task type with a generous ceiling, and record the completion tokens and the reasoning share. When the model reports it, `usage.completion_tokens_details.reasoning_tokens` gives the reasoning part; BroadNet’s docs list that field.

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])
MODEL = "broadnet/qwen3.8-27b"

def measure(prompts, ceiling=16_000, **kw):
    """Run real prompts with a generous ceiling and return per-call token counts."""
    rows = []
    for p in prompts:
        r = client.chat.completions.create(model=MODEL, max_tokens=ceiling,
                                           messages=[{"role": "user", "content": p}], **kw)
        u, d = r.usage, r.usage.completion_tokens_details
        reasoning = (d.reasoning_tokens or 0) if d else 0
        rows.append({"reasoning": reasoning, "answer": u.completion_tokens - reasoning,
                     "finish": r.choices[0].finish_reason})
    return rows

def pct(values, q):
    values = sorted(values)
    return values[min(len(values) - 1, int(q * len(values)))]

def recommend(rows, margin=1.2):
    total = [r["reasoning"] + r["answer"] for r in rows]
    return {"p50": pct(total, 0.5), "p95": pct(total, 0.95), "max": max(total),
            "max_tokens": int(pct(total, 0.95) * margin),
            "cut_off": sum(r["finish"] == "length" for r in rows)}

# print(recommend(measure(my_prompts)))                       # default effort
# print(recommend(measure(my_prompts, reasoning_effort="low")))
```

Rules for a number you can trust:

1.  **One task type per run.** Classification, extraction, code review and planning have very different distributions. One global `max_tokens` either wastes money on retries or lets runaway replies through.
2.  **50 or more real prompts.** The 95th percentile of ten samples is just the largest one.
3.  **Run it at each effort you use.** `reasoning_effort="low"` changes the distribution. BroadNet documents `low`; test other values before you rely on them.
4.  **If `cut_off` is not zero, raise the ceiling and run again.** A cut-off sample hides its true length.
5.  **Rerun when the model or prompt changes.** A new model version can think more or less.

Then set `max_tokens` per task from `recommend()`. The 95th percentile plus 20% leaves most calls well under the ceiling and sends only the rare long one to the retry path below. The free [reasoning token budget tool](https://broadrouter.com/tools/reasoning-token-budget/) does this arithmetic in your browser once you have your numbers.

## How does the context window limit the budget?

The context window holds the prompt and the answer together. BroadNet’s price feed defines `context_length` as “the most tokens one request can hold, prompt and answer together”. So:

**max\_tokens ≤ context length − prompt tokens**

| Model | Context | Room for max\_tokens with a 200,000-token prompt |
| --- | --- | --- |
| [GLM 5.3](https://broadrouter.com/models/glm-5.3/) | 1,048,576 | 848,576 |
| [DeepSeek V4.1 Flash](https://broadrouter.com/models/deepseek-v4.1-flash/) | 1,048,576 | 848,576 |
| Qwen3.8 27B | 262,144 | 62,144 |

This matters for long-context work. An agent that has grown its history to 250,000 tokens on Qwen3.8 27B has only about 12,000 tokens left for thinking and answer. Long-context tasks also tend to need more reasoning, so check the remaining room at the moment of the call, not once at startup. [Context window](https://broadrouter.com/glossary/context-window/) and the [context window calculator](https://broadrouter.com/tools/context-window-calculator/) cover the fit.

## What should you do when a reply is still cut off?

Check `finish_reason` on every call. If it is `length`, the answer is missing or incomplete, even when `content` has some text. Don’t parse it as if it were complete; a truncated JSON object is a common source of downstream errors ([structured output](https://broadrouter.com/blog/structured-output-json-schema/)).

Retry with a doubled budget, inside the room the context allows, up to a hard cap:

```python
import json, urllib.request

FEED = json.load(urllib.request.urlopen("https://inference.broadnet.ai/pricing.json"))
CONTEXT = {m["id"]: m.get("context_length") for m in FEED["models"]}

def ask(model, messages, budget=4_000, hard_cap=32_000, **kw):
    """Retry a cut-off reply with a doubled budget, never past the cap or the context window."""
    while True:
        r = client.chat.completions.create(model=model, messages=messages,
                                           max_tokens=budget, **kw)
        if r.choices[0].finish_reason != "length":
            return r
        room = CONTEXT[model] - r.usage.prompt_tokens        # context holds prompt + answer
        nxt = min(budget * 2, hard_cap, room)
        if nxt <= budget:
            raise RuntimeError(f"still cut off at max_tokens={budget}")
        print(f"cut off at {budget}, retrying with {nxt}")
        budget = nxt
```

Two alternatives to doubling, depending on the task:

-   **Retry at `reasoning_effort="low"`** with the same budget. For simple tasks the model often over-thinks, and less effort fixes it without a bigger ceiling.
-   **Split the task.** If a reply keeps hitting a high cap, the request is usually asking for too much at once.

Log every cut-off with the task type. A rising rate means the prompt, the model or the input data changed, and the budget needs measuring again. Retries are new inferences and are billed again, so keep the attempts bounded, as in [long-running agents: timeouts and retries](https://broadrouter.com/blog/long-running-agents-timeouts-retries/).

## How does streaming change the picture?

With `stream=True`, the reasoning happens before the first visible token, so a reasoning model can look stalled at the start. In a chat UI, show a “thinking” state until content arrives, and size client timeouts for the thinking time, not just the answer.

Some servers also stream the reasoning text itself, in a separate field. The name is not standard: DeepSeek’s own API uses `reasoning_content`, and vLLM renamed its field from `reasoning_content` to `reasoning`, warning that old client code “could silently read an empty” field. BroadNet’s docs don’t document a reasoning text field, so don’t build features on one without checking the raw response for your model. If you do read it, read it defensively:

```python
for chunk in stream:
    if chunk.choices:
        delta = chunk.choices[0].delta
        thinking = getattr(delta, "reasoning", None) or getattr(delta, "reasoning_content", None)
        # log or display `thinking` only if your provider actually sends it
        if delta.content:
            print(delta.content, end="")
    elif chunk.usage:                                  # last chunk, with include_usage
        print("\nreasoning tokens:", getattr(chunk.usage.completion_tokens_details, "reasoning_tokens", None))
```

The usage in the last chunk needs `stream_options={"include_usage": True}`, which BroadNet documents. More on streams in [the streaming guide](https://broadrouter.com/guides/streaming-chat-completions/) and [SSE explained](https://broadrouter.com/blog/llm-streaming-sse-explained/).

## Which parameter name should you send?

| API | Parameter | Includes reasoning |
| --- | --- | --- |
| OpenAI Chat Completions | `max_completion_tokens` (`max_tokens` deprecated) | Yes |
| OpenAI Responses | `max_output_tokens` | Yes |
| BroadNet Inference (Chat Completions) | `max_tokens`, as documented | Yes |
| vLLM servers | `max_tokens`, plus optional `thinking_token_budget` | Yes |

When moving code between providers, send the name the provider documents, and confirm with one test call that the limit takes effect. A limit silently ignored looks fine until a runaway reply arrives. The [migration checklist](https://broadrouter.com/blog/openai-compatible-api-migration-checklist/) covers the other parameters that change.

## Where BroadNet Inference fits

BroadNet’s three reasoning models report reasoning tokens in `usage` when the model provides them, so the measurement above works as written. DeepSeek V4.1 Flash and GLM 5.3 give the most room, with 1,048,576 tokens of context. Qwen3.8 27B has 262,144. If you don’t need reasoning for a task, [Gemma 4 VL 26B](https://broadrouter.com/models/gemma-4-vl-26b/) is not listed as a reasoning model and avoids the budget question altogether. On a flat [DeepSeek V4.1 Flash plan](https://broadrouter.com/guides/unlimited-llm-api-sessions/), thinking tokens carry no per-token charge, but cut-off replies still cost time, so the same budgeting applies. BroadNet doesn’t document a separate reasoning cap; if you need one, use a provider or self-hosted server that offers it.

Free tools for this

-   [OpenAI to BroadNet migration helper](https://broadrouter.com/tools/openai-to-broadnet-migration/) Paste OpenAI SDK code; get base URL, key and model edits, a diff and the checks that remain.
-   [max\_tokens calculator for reasoning models](https://broadrouter.com/tools/reasoning-token-budget/) Answer length + reasoning budget + margin = a max\_tokens that does not cut the answer off.
-   [JSON schema generator for LLM structured output](https://broadrouter.com/tools/json-schema-generator/) Sample JSON to strict schema, Pydantic, Zod and a ready request.
-   [LLM JSON output validator](https://broadrouter.com/tools/llm-json-output-validator/) Parse errors by line and column, safe repairs, cut-off detection and JSON Schema checks for model output.

[All free tools](https://broadrouter.com/tools/)

## Terms in this article

[Token](https://broadrouter.com/glossary/token/)

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

[Context window](https://broadrouter.com/glossary/context-window/)

A context window is the maximum number of tokens a language model can work with in a single request. It covers the prompt, conversation history, tool definitions, images and the generated output, including any reasoning tokens. Anything that does not fit must be cut, summarised or split across several requests.

[Input and output tokens](https://broadrouter.com/glossary/input-and-output-tokens/)

Input tokens are the tokens a language model reads in a request, including the system prompt, conversation history, tool definitions and images. Output tokens are the tokens it generates in reply, including any reasoning. LLM APIs usually price them separately, and output usually costs more.

[Reasoning tokens](https://broadrouter.com/glossary/reasoning-tokens/)

Reasoning tokens are tokens a reasoning model generates to work through a problem before it writes its visible answer. They are usually billed as output tokens, count toward the output limit and the context window, and may be hidden or summarized by the API.

[max\_tokens](https://broadrouter.com/glossary/max-tokens/)

max\_tokens is an LLM API parameter that sets the most tokens a model may generate in one response. When generation reaches the limit, it stops and the response is marked as cut off, for example with finish\_reason set to "length".

[Chat Completions API](https://broadrouter.com/glossary/chat-completions-api/)

The Chat Completions API is OpenAI's HTTP endpoint, POST /v1/chat/completions, that takes a model name and a list of role-tagged messages and returns the model's next message. Its request and response shape became the common format that OpenAI-compatible providers and SDKs reuse.

[vLLM](https://broadrouter.com/glossary/vllm/)

vLLM is an open-source library for LLM inference and serving, first developed at UC Berkeley's Sky Computing Lab. It manages KV cache memory with PagedAttention, batches incoming requests continuously, supports many quantization formats and exposes an OpenAI-compatible HTTP server.

[Streaming](https://broadrouter.com/glossary/streaming/)

Streaming is a way of returning a language model response in small chunks while it is still being generated, instead of as one reply at the end. OpenAI-compatible APIs stream over server-sent events when the request sets stream to true.

## Questions

### Why does my reasoning model return an empty response?

The model used the whole max\_tokens budget on hidden reasoning before it wrote the answer. finish\_reason is then "length". Raise max\_tokens, lower reasoning\_effort, or both. A short guide with the quick fix is linked from this post.

### What max\_tokens should I set for a reasoning model?

Measure first. Run 50 or more real prompts with a generous ceiling, take the 95th percentile of completion\_tokens for that task, and add about 20%. BroadNet's docs give 1,000 tokens or more as a floor; OpenAI suggests reserving 25,000 when you start out on its models.

### Does a higher max\_tokens cost more?

Not by itself. You are billed for the tokens the model generates, not for the ceiling. A higher ceiling costs more only when a reply would otherwise have been cut off, and then a cut-off reply would have cost you a retry anyway.

### Is max\_completion\_tokens the same as max\_tokens?

On OpenAI, max\_completion\_tokens replaces the deprecated max\_tokens and explicitly includes reasoning tokens. Many OpenAI-compatible providers document max\_tokens instead. BroadNet's docs use max\_tokens and say reasoning counts toward it.

### Can I cap reasoning separately from the answer?

Only where the server supports it. vLLM has a thinking\_token\_budget parameter, and some providers have their own. BroadNet documents reasoning\_effort set to low, not a separate reasoning cap, so budget the two together.

## Keep reading

-   [Reasoning tokens: why you pay for more output than you see](https://broadrouter.com/blog/reasoning-tokens-cost/)
-   [LLM streaming explained: SSE chunks, deltas, usage and cancel](https://broadrouter.com/blog/llm-streaming-sse-explained/)
-   [JSON mode vs json\_schema: structured output that holds up](https://broadrouter.com/blog/structured-output-json-schema/)
-   [Long-running agents: timeouts, retries and checkpoints that hold](https://broadrouter.com/blog/long-running-agents-timeouts-retries/)
-   [Switching to an OpenAI-compatible API: a migration checklist](https://broadrouter.com/blog/openai-compatible-api-migration-checklist/)
-   [A private endpoint starts with a clear workload](https://broadrouter.com/blog/private-endpoint-planning/)
-   [Before fine-tuning: define the problem to solve](https://broadrouter.com/blog/fine-tuning-readiness/)
-   [Guide: Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/)
-   [Guide: Stream chat completions and handle dropped connections](https://broadrouter.com/guides/streaming-chat-completions/)
-   [Guide: Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-reasoning-model-max-tokens-budget&utm_content=page-cta) ·

**Sources**

-   [BroadNet Inference API reference: reasoning counts toward max\_tokens, 1,000 tokens or more, reasoning\_effort low, usage fields (checked 2026-11-15)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-reasoning-model-max-tokens-budget&utm_content=sources)
-   [BroadNet pricing.json field reference: context\_length holds prompt and answer together (checked 2026-11-15)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-reasoning-model-max-tokens-budget&utm_content=sources#pricing-json)
-   [OpenAI reasoning guide: incomplete responses, reserve at least 25,000 tokens when starting out (checked 2026-11-15)](https://developers.openai.com/api/docs/guides/reasoning)
-   [OpenAI Chat Completions reference: max\_completion\_tokens includes reasoning tokens; max\_tokens deprecated (checked 2026-11-15)](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create)
-   [DeepSeek API docs, thinking mode: reasoning\_content field, effort levels (checked 2026-11-15)](https://api-docs.deepseek.com/guides/thinking_mode)
-   [vLLM docs, reasoning outputs: reasoning field (formerly reasoning\_content), thinking\_token\_budget (checked 2026-11-15)](https://docs.vllm.ai/en/latest/features/reasoning_outputs.html)
