BroadRouter

Use Pydantic AI with BroadNet Inference (OpenAICompatibleProvider)

Updated

Install pydantic-ai-slim[openai], then build OpenAIChatModel with a BroadNet model ID such as broadnet/glm-5.3 and an OpenAICompatibleProvider whose base URL is https://inference.broadnet.ai/v1 and whose API key is your BroadNet key. Pass the model to Agent. Use OpenAIChatModel, not the Responses model, because BroadNet serves chat completions. Pydantic AI's docs say OpenAIProvider with base_url is meant for OpenAI itself.

Pydantic AI is a Python agent framework from the Pydantic team. For a service it has no built-in provider for, its docs say to use OpenAICompatibleProvider with the service’s base_url. BroadNet Inference serves standard chat completions at https://inference.broadnet.ai/v1, so that is the route. BroadRouter is BroadNet’s own site; this follows the Pydantic AI docs as of version 2.68.

Install

pip install "pydantic-ai-slim[openai]"
export BROADNET_API_KEY='your-broadnet-key'

Connect

import os
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai_compatible import OpenAICompatibleProvider

model = OpenAIChatModel(
    'broadnet/glm-5.3',
    provider=OpenAICompatibleProvider(
        'broadnet',
        base_url='https://inference.broadnet.ai/v1',
        api_key=os.environ['BROADNET_API_KEY'],
    ),
)

agent = Agent(model)
result = agent.run_sync('Give me three names for a Go linting tool.')
print(result.output)

Running this is the test: a printed answer means the key, URL and model ID are right.

  • The first argument is the model ID sent to BroadNet. Use it exactly, with the broadnet/ prefix.
  • The provider name ('broadnet' here) is recorded on every response. Pydantic AI’s docs say to keep it stable, because it decides which data from earlier responses is sent back to the service.
  • Stay on OpenAIChatModel. Pydantic AI’s docs say to use OpenAIResponsesModel only if the service implements the Responses API. BroadNet doesn’t list it.

Tools and structured output

from pydantic import BaseModel

class City(BaseModel):
    name: str
    country: str

agent = Agent(model, output_type=City)
print(agent.run_sync('Which city hosts the Burj Khalifa?').output)

Use broadnet/glm-5.3 or broadnet/qwen3.8-27b for tools and structured output; BroadNet’s docs list both features for these two models only.

When to set a model profile

Pydantic AI picks a request profile from the model name. If a feature is ignored or rejected, its docs say to pass your own OpenAIModelProfile. Two settings are worth knowing for BroadNet:

from pydantic_ai.profiles.openai import OpenAIModelProfile

model = OpenAIChatModel(
    'broadnet/glm-5.3',
    provider=OpenAICompatibleProvider('broadnet', base_url='https://inference.broadnet.ai/v1', api_key=os.environ['BROADNET_API_KEY']),
    profile=OpenAIModelProfile(
        openai_chat_supports_max_completion_tokens=False,  # send max_tokens, the field BroadNet documents
        openai_supports_strict_tool_definition=False,      # only if strict tool schemas are rejected
    ),
)

Pass the output limit through model settings, for example agent.run_sync(prompt, model_settings={'max_tokens': 2000}). Reasoning counts toward it on DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B.

Which model to pick

Agent Model
Tools, structured output broadnet/glm-5.3
Images or Arabic text, with tools broadnet/qwen3.8-27b
Long documents, flat-price plan broadnet/deepseek-v4.1-flash
Free starting credit broadnet/gemma-4-26b-a4b (32,768-token context)

Prices for every model.

Troubleshooting

  • 401. The key is wrong, expired or out of allowance.
  • 404 or model not found. base_url must end in /v1. Check the model ID. Paid models need a top-up; the $5 credit covers only broadnet/gemma-4-26b-a4b and the two Transcribe models.
  • Empty output or a validation retry loop. Reasoning used the output budget. Raise max_tokens. Why answers come back empty.
  • 400 on tool definitions. Set openai_supports_strict_tool_definition=False in the profile, and use a model BroadNet lists with tools.
  • 429 when you run agents concurrently. A pay-as-you-go key allows 30 requests a minute and 4 in flight. See errors and retries.

Questions

Should I use OpenAIProvider or OpenAICompatibleProvider for BroadNet?

OpenAICompatibleProvider. Pydantic AI's docs say OpenAIProvider also takes a base_url, but it is for OpenAI itself or a transparent proxy to it, and it assumes OpenAI's models and features.

Why use OpenAIChatModel and not the default OpenAI model?

Pydantic AI's default OpenAI model uses the Responses API. BroadNet serves chat completions at /v1/chat/completions, which is what OpenAIChatModel calls.

Do tools and structured output work?

On models BroadNet lists with tools and structured output, GLM 5.3 and Qwen3.8 27B, yes. BroadNet's docs don't list tools for DeepSeek V4.1 Flash, so test agents that use tools on it first.

How do I stop Pydantic AI sending max_completion_tokens?

Pass profile=OpenAIModelProfile(openai_chat_supports_max_completion_tokens=False) to OpenAIChatModel. Pydantic AI then sends the older max_tokens field, which is the one BroadNet's API reference describes.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity