BroadRouter

Chain-of-thought (CoT)

Updated

Chain-of-thought (CoT) is a prompting technique in which a language model is shown or asked to write intermediate reasoning steps before its final answer. It improves accuracy on multi-step problems such as arithmetic and logic, and reasoning models now generate it automatically.

Also called: CoT, chain-of-thought prompting, step-by-step prompting, CoT prompting

Chain-of-thought started as a prompt trick and became a model feature.

How it works

Wei and colleagues published the method in 2022. Instead of few-shot examples that show only question and answer, they wrote examples that show the steps in between. With eight such examples, a 540-billion-parameter model reached state-of-the-art accuracy on GSM8K, a set of grade-school maths word problems, ahead of a fine-tuned GPT-3 with a verifier. They tested arithmetic, commonsense and symbolic reasoning.

A chain-of-thought example in a prompt:

Q: A shop has 23 apples. It uses 20 and buys 6 more. How many are left?
A: It starts with 23. After using 20 it has 3. After buying 6 it has 9. The answer is 9.

Writing the steps gives the model more tokens of computation before it commits to an answer, and each step conditions the next. It is a form of few-shot prompting where the examples teach a method, not just a format.

How reasoning models internalize it

Reasoning models are trained to produce this step-by-step text themselves. OpenAI’s guide says they “use internal reasoning tokens before producing a response”, and that those tokens are not visible through its API but still use context and are billed as output. Some open models return the reasoning in a separate field instead. See reasoning tokens for how they are counted.

Why it matters

  • Accuracy. Multi-step tasks such as maths, planning and code fixes benefit most. Simple lookups gain little.
  • Cost and latency. Every step is an output token. A long chain can cost more than the answer.
  • Not proof. Written steps can look sound and still be wrong, so they do not prevent hallucination.

On BroadNet Inference

BroadNet’s docs say DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason before they answer, and the reasoning counts toward max_tokens. Leave 1,000 tokens or more, or set reasoning_effort to low; with a small limit the answer can come back empty. When reported, usage.completion_tokens_details.reasoning_tokens shows how many were used. The docs do not list BroadNet Gemma 4 VL 26B as a reasoning model, so classic chain-of-thought prompting is the way to ask it for steps. The empty-response guide and the post on reasoning token cost cover budgeting.

Questions

Is chain-of-thought the same as reasoning tokens?

Related, not the same. Chain-of-thought is the technique of writing out steps. Reasoning tokens are what a reasoning model generates when it does this internally, and they are billed as output.

Should I write "think step by step" for a reasoning model?

Usually it is unnecessary. Reasoning models already reason before answering. OpenAI's guide suggests giving them a clear goal and constraints rather than prescribing every step.

Does chain-of-thought cost more?

Yes. The steps are output tokens, which are usually the most expensive tokens, and they add latency.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity