Every token a model writes is picked from a probability distribution. Logprobs expose that distribution, one token at a time.
How it works
At each step the model scores every token in its vocabulary and turns the scores into probabilities. The API can report the natural log of the probability of the token it chose. Logs are used because the raw numbers get very small; adding logs is the same as multiplying probabilities.
In OpenAI’s Chat Completions API you ask for them with two request fields:
logprobs: truereturns a log probability for each output token.top_logprobsis “an integer between 0 and 20” for how many likely alternatives to list at each position. It requireslogprobs: true.
Each entry in choices[0].logprobs.content holds the token, its logprob, its UTF-8 bytes and a top_logprobs list. Convert with exp(logprob):
| logprob | probability |
|---|---|
| 0 | 1.00 |
| -0.105 | about 0.90 |
| -0.693 | about 0.50 |
| -2.303 | about 0.10 |
Why it matters
OpenAI’s cookbook lists practical uses:
- Classification confidence. Ask for a one-word label and read the logprob of that token. Send low-confidence items to a human.
- Retrieval checks. Ask whether the supplied context answers the question, and use the probability of “yes” as a score. The cookbook links this to reducing hallucination.
- Autocomplete. Only suggest the next word when the model is confident.
- Perplexity. Average the logprobs of a whole answer, negate the result and take e to that power. Lower perplexity means the model found the text more predictable.
Logprobs also make sampling visible. Raising temperature or top-p lets lower-probability tokens through, and the logprobs show it. They are a cheap complement to an LLM-as-judge step, since they come with the answer at no extra request.
On BroadNet Inference
BroadNet’s API reference does not document logprobs or top_logprobs for any model. Do not assume an OpenAI-compatible endpoint returns them. Send one request with logprobs: true, check whether choices[0].logprobs is filled, and plan a fallback, such as a judge prompt, if it is not. The Python SDK guide shows the client setup for a test.
Common mistakes
- Reading a high logprob as proof of a correct fact. It only means the model expected that token.
- Comparing logprobs across different models. Tokenizers and calibration differ, so the numbers are not on the same scale.