BroadRouter
Get started

Structured output

Updated

Structured output is an LLM API feature that constrains a model's reply to a format the developer specifies, usually a JSON Schema, so code can parse it reliably. It is stricter than JSON mode, which guarantees valid JSON but not a particular shape.

Also called: structured outputs, JSON schema output, response_format

Code needs predictable input. Structured output turns a model’s free text into data your program can trust.

How it works

There are three levels of control:

Method What you get
Ask in the prompt Usually JSON, sometimes with extra text or missing fields
JSON mode Valid JSON, any shape
Structured output with a schema JSON that matches your schema

In the OpenAI format, JSON mode is response_format: {"type": "json_object"}. Structured output passes a schema:

"response_format": {
  "type": "json_schema",
  "json_schema": {
    "name": "invoice",
    "strict": true,
    "schema": {
      "type": "object",
      "properties": {
        "vendor": { "type": "string" },
        "total": { "type": "number" }
      },
      "required": ["vendor", "total"],
      "additionalProperties": false
    }
  }
}

OpenAI’s reference says the json_schema type “ensures the model will match your supplied JSON schema.” Many providers do this with constrained decoding: restricting which tokens the model may produce at each step, so output that breaks the schema cannot be generated.

Why it matters

  • Fewer parse errors. Extraction pipelines, classifiers and agents can read fields directly.
  • Simpler prompts. The schema replaces long format instructions.
  • Not a full guarantee. OpenAI’s guide says your app should still handle refusals, incomplete responses and errors. A reply cut off by max_tokens is broken JSON whatever the schema says.

Support varies across OpenAI-compatible APIs. Some accept only json_schema, some only json_object, and some ignore the field. Check before you depend on it.

On BroadNet Inference

BroadNet’s docs list structured output for GLM 5.3 and Qwen3.8 27B. The docs do not name the request parameter, so test response_format on your model, and keep a JSON parser with error handling either way. The LLM JSON output validator shows where a reply breaks and checks it against your schema. Both models reason first, so give max_tokens room for thinking plus the JSON.

Structured output pairs well with vision models. The Arabic invoice guide sends an image and asks for the fields as JSON. For actions rather than answers, use tool calling.

Common mistakes

  • Forgetting additionalProperties: false, or not listing every field in required, when using strict mode.
  • Not checking finish_reason before parsing.

Questions

What is the difference between JSON mode and structured output?

JSON mode only promises that the reply parses as JSON. Structured output with a schema also promises the fields, types and required keys you defined. OpenAI's guide says only Structured Outputs ensure schema adherence.

Do I still need to validate the output?

Yes. A reply can still be a refusal or be cut off by max_tokens, and a schema cannot express every business rule, such as a total that must equal the sum of line items.

Should I use structured output or tool calling?

Use structured output when you want the final answer in a fixed shape. Use tool calling when the model should ask your code to do something and then continue.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity