BroadRouter
Get started

Stop sequences

Updated

Stop sequences are strings you send with a request that make the model stop generating as soon as it produces one of them. OpenAI calls the field stop and accepts up to 4 sequences; Anthropic calls it stop_sequences. The matched text is normally left out of the returned output.

Also called: stop sequence, stop parameter, stop strings, stop_sequences

Sometimes you know exactly where an answer should end. A stop sequence tells the server to cut generation there instead of paying for tokens you will throw away.

How it works

You pass one or more strings. After each new token, the server checks whether the output now contains one of them. If it does, generation ends.

{
  "model": "broadnet/gemma-4-26b-a4b",
  "messages": [{"role": "user", "content": "List three fruits, one per line, then write END."}],
  "stop": ["END"]
}

Provider details from the sources:

Provider Field Notes
OpenAI Chat Completions stop “Up to 4 sequences”; the reference still says “Not supported with latest reasoning models o3 and o4-mini” (wording as of 2026-11-16)
Anthropic Messages stop_sequences Response has stop_reason: "stop_sequence" and names the match in stop_sequence
vLLM (open-model servers) stop “String(s) that stop the generation when they are generated”; output excludes them

In the OpenAI format a matched stop sequence reports finish_reason stop, the same value as a natural end. Anthropic separates the two.

Why it matters

  • Cost and latency. Output tokens are billed and are slow to generate. Stopping at a known marker saves both.
  • Format control. Few-shot prompts often use a delimiter such as ### between examples. A stop on that delimiter stops the model from writing the next example itself.
  • Limits. A stop sequence cannot shape what comes before the cut. For machine-readable output, structured output is the stronger tool. Keep max_tokens as the hard ceiling.

On BroadNet Inference

BroadNet’s API reference does not document stop. Test it before relying on it: send a prompt that should hit your marker and check whether the reply ends there. As a fallback, cut the text in your client at the first marker; in a stream you can also close the reader once the marker appears. Reasoning models on BroadNet (DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B) think before they answer, so test whether a stop string can match inside the reasoning. The migration walkthrough lists other OpenAI fields to check.

Common mistakes

  • Choosing a stop string that can appear in normal output, such as a newline or a common word.
  • Assuming the stop string comes back in the reply and parsing for it.

Questions

How many stop sequences can I send?

OpenAI's reference allows up to 4 sequences in stop. Other providers set their own limits, so check their reference.

Is the stop sequence included in the reply?

Usually not. vLLM's documentation says the returned output excludes the stop string. Anthropic returns which sequence fired in a separate stop_sequence field.

Does BroadNet Inference support stop?

BroadNet's API reference does not document stop. Test it on your model before relying on it, and keep a client-side cut as a backup.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity