BroadRouter
Get started

Get structured output with Instructor on BroadNet Inference

Updated

Wrap an OpenAI client that has base_url https://inference.broadnet.ai/v1 and your BroadNet key with instructor.from_openai, then call chat.completions.create with response_model set to your Pydantic class and model set to broadnet/glm-5.3 or broadnet/qwen3.8-27b. Those are the two models BroadNet lists with tools and structured output. Mode.TOOLS is the default; Mode.JSON_SCHEMA also works.

Instructor is a Python library that turns LLM replies into validated Pydantic objects and can retry when validation fails. It patches the OpenAI client, and its docs cover OpenAI-compatible servers through base_url. BroadNet Inference serves chat completions at https://inference.broadnet.ai/v1, and lists tools and structured output for GLM 5.3 and Qwen3.8 27B. BroadRouter is BroadNet’s own site; this follows Instructor’s docs as of version 1.17.0.

Install

pip install instructor
export BROADNET_API_KEY='your-broadnet-key'

Extract an object

import os
import instructor
from openai import OpenAI
from pydantic import BaseModel

class Invoice(BaseModel):
    vendor: str
    total: float
    currency: str

client = instructor.from_openai(
    OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"]),
    mode=instructor.Mode.TOOLS,
)

invoice = client.chat.completions.create(
    model="broadnet/glm-5.3",
    response_model=Invoice,
    max_tokens=4000,
    messages=[{"role": "user", "content": "Acme Trading invoice, total 1,250.00 SAR"}],
)
print(invoice)

Instructor turns Invoice into a tool schema, forces the model to call it with tool_choice, and validates the arguments. Pass max_retries to control how many times it asks again, with the validation error, when the object doesn’t validate.

Or use from_provider

pclient = instructor.from_provider(
    "openai/broadnet/glm-5.3",
    base_url="https://inference.broadnet.ai/v1",
    api_key=os.environ["BROADNET_API_KEY"],
)
invoice = pclient.create(response_model=Invoice, max_tokens=4000,
                         messages=[{"role": "user", "content": "Acme Trading, 1,250.00 SAR"}])

The openai/ part picks the client. The rest, broadnet/glm-5.3, is sent as the model ID.

Which mode

Mode What it sends Use with BroadNet
Mode.TOOLS (default) A forced tool call with your schema Yes, on GLM 5.3 or Qwen3.8 27B
Mode.JSON_SCHEMA response_format of type json_schema Yes, on GLM 5.3 or Qwen3.8 27B
Mode.MD_JSON A plain request; JSON parsed from the text Fallback for other models; test first
Mode.RESPONSES_TOOLS The Responses API No: BroadNet serves chat completions

Images

Qwen3.8 27B takes images as well as tools, so it can read a photo or scan straight into your model:

import base64
img = base64.b64encode(open("invoice.jpg", "rb").read()).decode()
invoice = client.chat.completions.create(
    model="broadnet/qwen3.8-27b",
    response_model=Invoice,
    max_tokens=4000,
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "Extract the invoice fields."},
        {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64," + img}},
    ]}],
)

On a pay-as-you-go key an image counts as input tokens. See reading Arabic invoices with vision.

How we tested this

On 2026-11-15 we installed instructor 1.17.0 with openai 3.3.0 in a clean virtual environment and ran the examples above against a local mock server that speaks BroadNet’s chat-completions shape. We read the logged requests. Mode.TOOLS sent the schema as a tool with a forced tool_choice, Mode.JSON_SCHEMA sent a json_schema response format, and both returned a valid Invoice. from_provider sent the model ID unchanged. Our mock doesn’t write free-text JSON, so we did not test Mode.MD_JSON, and we did not send the image example. We had no BroadNet key, so we did not test real model output.

Troubleshooting

  • Validation retries that never succeed. Reasoning counts toward max_tokens on GLM 5.3 and Qwen3.8 27B. A low limit can cut off the tool arguments. Raise it. Why answers come back empty.
  • 400 on the request. Check you are on GLM 5.3 or Qwen3.8 27B. If a forced tool_choice is rejected, try Mode.JSON_SCHEMA.
  • 401. The key is wrong, expired or out of allowance.
  • 404. base_url must end in /v1. GLM 5.3 and Qwen3.8 27B need a top-up; the $5 credit doesn’t cover them.
  • 429 in batch extraction. A pay-as-you-go key allows 30 requests a minute and 4 in flight. Each validation retry is another request. See errors and retries.

Questions

Which Instructor mode should I use with BroadNet?

Mode.TOOLS, the default, on GLM 5.3 or Qwen3.8 27B, which BroadNet lists with tools and structured output. Mode.JSON_SCHEMA sends a json_schema response_format instead. Both produced valid objects in our mock-server test.

Can I use Mode.RESPONSES_TOOLS?

No. Instructor's docs say not to pick it unless the server implements the Responses API. BroadNet serves chat completions only.

Does Instructor work with DeepSeek V4.1 Flash or Gemma on BroadNet?

BroadNet doesn't list tools or structured output for those models. Mode.MD_JSON asks for JSON in plain text and parses it, which may work, but test it on your own data before relying on it.

How do I write the model string for from_provider?

Use openai/broadnet/glm-5.3. Instructor splits off the first segment as the provider, and in our test it sent broadnet/glm-5.3 to the server unchanged.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity