Chain-of-thought started as a prompt trick and became a model feature.
How it works
Wei and colleagues published the method in 2022. Instead of few-shot examples that show only question and answer, they wrote examples that show the steps in between. With eight such examples, a 540-billion-parameter model reached state-of-the-art accuracy on GSM8K, a set of grade-school maths word problems, ahead of a fine-tuned GPT-3 with a verifier. They tested arithmetic, commonsense and symbolic reasoning.
A chain-of-thought example in a prompt:
Q: A shop has 23 apples. It uses 20 and buys 6 more. How many are left?
A: It starts with 23. After using 20 it has 3. After buying 6 it has 9. The answer is 9.
Writing the steps gives the model more tokens of computation before it commits to an answer, and each step conditions the next. It is a form of few-shot prompting where the examples teach a method, not just a format.
How reasoning models internalize it
Reasoning models are trained to produce this step-by-step text themselves. OpenAI’s guide says they “use internal reasoning tokens before producing a response”, and that those tokens are not visible through its API but still use context and are billed as output. Some open models return the reasoning in a separate field instead. See reasoning tokens for how they are counted.
Why it matters
- Accuracy. Multi-step tasks such as maths, planning and code fixes benefit most. Simple lookups gain little.
- Cost and latency. Every step is an output token. A long chain can cost more than the answer.
- Not proof. Written steps can look sound and still be wrong, so they do not prevent hallucination.
On BroadNet Inference
BroadNet’s docs say DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason before they answer, and the reasoning counts toward max_tokens. Leave 1,000 tokens or more, or set reasoning_effort to low; with a small limit the answer can come back empty. When reported, usage.completion_tokens_details.reasoning_tokens shows how many were used. The docs do not list BroadNet Gemma 4 VL 26B as a reasoning model, so classic chain-of-thought prompting is the way to ask it for steps. The empty-response guide and the post on reasoning token cost cover budgeting.