BroadRouter
Get started

Logprobs

Updated

Logprobs are the log probabilities a language model assigned to the tokens it generated. An API that returns them shows, for each output token, how likely the model judged it and, optionally, the most likely alternative tokens at that position.

Also called: log probabilities, token log probabilities, top_logprobs, token probabilities

Every token a model writes is picked from a probability distribution. Logprobs expose that distribution, one token at a time.

How it works

At each step the model scores every token in its vocabulary and turns the scores into probabilities. The API can report the natural log of the probability of the token it chose. Logs are used because the raw numbers get very small; adding logs is the same as multiplying probabilities.

In OpenAI’s Chat Completions API you ask for them with two request fields:

  • logprobs: true returns a log probability for each output token.
  • top_logprobs is “an integer between 0 and 20” for how many likely alternatives to list at each position. It requires logprobs: true.

Each entry in choices[0].logprobs.content holds the token, its logprob, its UTF-8 bytes and a top_logprobs list. Convert with exp(logprob):

logprob probability
0 1.00
-0.105 about 0.90
-0.693 about 0.50
-2.303 about 0.10

Why it matters

OpenAI’s cookbook lists practical uses:

  • Classification confidence. Ask for a one-word label and read the logprob of that token. Send low-confidence items to a human.
  • Retrieval checks. Ask whether the supplied context answers the question, and use the probability of “yes” as a score. The cookbook links this to reducing hallucination.
  • Autocomplete. Only suggest the next word when the model is confident.
  • Perplexity. Average the logprobs of a whole answer, negate the result and take e to that power. Lower perplexity means the model found the text more predictable.

Logprobs also make sampling visible. Raising temperature or top-p lets lower-probability tokens through, and the logprobs show it. They are a cheap complement to an LLM-as-judge step, since they come with the answer at no extra request.

On BroadNet Inference

BroadNet’s API reference does not document logprobs or top_logprobs for any model. Do not assume an OpenAI-compatible endpoint returns them. Send one request with logprobs: true, check whether choices[0].logprobs is filled, and plan a fallback, such as a judge prompt, if it is not. The Python SDK guide shows the client setup for a test.

Common mistakes

  • Reading a high logprob as proof of a correct fact. It only means the model expected that token.
  • Comparing logprobs across different models. Tokenizers and calibration differ, so the numbers are not on the same scale.

Questions

How do I turn a logprob into a percentage?

Take e to the power of the logprob. A logprob of -0.105 is about 0.90, or 90%. A logprob of 0 means the model gave that token probability 1.

Are logprobs a measure of whether the answer is true?

No. They measure how confident the model was in its wording, not whether the fact is correct. A model can be confidently wrong.

Does BroadNet Inference return logprobs?

BroadNet's API reference does not document logprobs. Send a test request with logprobs set to true and check the response before you build on it.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity