Sampling is random on purpose. A seed fixes the randomness so a rerun has a chance of matching the first run.
How it works
When temperature is above zero, the server draws each next token at random from the model’s distribution. A random number generator drives those draws. Fix its starting value, the seed, and the same request should take the same draws.
OpenAI’s reference marks seed as Beta. The cookbook says “our system will make a best effort to sample deterministically”. To get matching outputs it says to keep these the same across requests:
- the
seedinteger, - the prompt and messages,
temperature, top_p andmax_tokens.
The response includes system_fingerprint, which OpenAI says “represents the backend configuration that the model runs with”. If it changes between two runs, expect different outputs even with the same seed.
r = client.chat.completions.create(
model=MODEL, seed=42, temperature=0.7,
messages=[{"role": "user", "content": "Name a colour."}],
)
print(r.choices[0].message.content, r.system_fingerprint)
Open-model servers have the same knob. vLLM’s SamplingParams has seed, “Random seed to use for the generation”, with a default of none.
Why it matters
- Evals. When you compare prompts or models in an eval, a fixed seed reduces noise between runs. Still run several samples; best effort is not a guarantee.
- Debugging. A seeded request makes a strange reply easier to reproduce.
- Caching. If you cache responses, include the seed in the cache key, as the response caching post does.
On BroadNet Inference
BroadNet’s API reference does not document seed or system_fingerprint. Test it before relying on it: send the same seeded request five times and compare the outputs. If they differ, lower the temperature, store the outputs you need, and evaluate on several samples rather than one. The migration walkthrough lists seed among the OpenAI fields to test or remove.
Common mistakes
- Treating a seed as a promise of byte-identical output.
- Changing max_tokens or the system prompt between runs and blaming the seed for the difference.