Sometimes you know exactly where an answer should end. A stop sequence tells the server to cut generation there instead of paying for tokens you will throw away.
How it works
You pass one or more strings. After each new token, the server checks whether the output now contains one of them. If it does, generation ends.
{
"model": "broadnet/gemma-4-26b-a4b",
"messages": [{"role": "user", "content": "List three fruits, one per line, then write END."}],
"stop": ["END"]
}
Provider details from the sources:
| Provider | Field | Notes |
|---|---|---|
| OpenAI Chat Completions | stop |
“Up to 4 sequences”; the reference still says “Not supported with latest reasoning models o3 and o4-mini” (wording as of 2026-11-16) |
| Anthropic Messages | stop_sequences |
Response has stop_reason: "stop_sequence" and names the match in stop_sequence |
| vLLM (open-model servers) | stop |
“String(s) that stop the generation when they are generated”; output excludes them |
In the OpenAI format a matched stop sequence reports finish_reason stop, the same value as a natural end. Anthropic separates the two.
Why it matters
- Cost and latency. Output tokens are billed and are slow to generate. Stopping at a known marker saves both.
- Format control. Few-shot prompts often use a delimiter such as
###between examples. A stop on that delimiter stops the model from writing the next example itself. - Limits. A stop sequence cannot shape what comes before the cut. For machine-readable output, structured output is the stronger tool. Keep max_tokens as the hard ceiling.
On BroadNet Inference
BroadNet’s API reference does not document stop. Test it before relying on it: send a prompt that should hit your marker and check whether the reply ends there. As a fallback, cut the text in your client at the first marker; in a stream you can also close the reader once the marker appears. Reasoning models on BroadNet (DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B) think before they answer, so test whether a stop string can match inside the reasoning. The migration walkthrough lists other OpenAI fields to check.
Common mistakes
- Choosing a stop string that can appear in normal output, such as a newline or a common word.
- Assuming the stop string comes back in the reply and parsing for it.