BroadRouter
Get started

Use the OpenAI Python SDK with BroadNet Inference

Updated

Install the openai package, then create the client with base_url set to https://inference.broadnet.ai/v1 and your BroadNet key as api_key. Every chat.completions call works unchanged; only the model ID changes, for example broadnet/deepseek-v4.1-flash. BroadNet serves chat completions and transcriptions, not the Responses API.

You need a BroadNet Inference account and an inference key. New accounts start with $5 of credit and no card; create one at inference.broadnet.ai. The console shows a key once, and only its hash is stored, so copy it straight into your environment.

1. Install and set the key

The current openai package (3.x) needs Python 3.10 or newer.

python -m pip install openai
export BROADNET_API_KEY='paste-the-key-once'

2. Make a first request

Save this as first_request.py:

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://inference.broadnet.ai/v1",
    api_key=os.environ["BROADNET_API_KEY"],
)

response = client.chat.completions.create(
    model="broadnet/gemma-4-26b-a4b",
    messages=[{"role": "user", "content": "Reply with one short sentence."}],
)
print(response.choices[0].message.content)

Run it with python first_request.py. This example uses broadnet/gemma-4-26b-a4b because it works on the free credit. The other chat models are available after a top-up.

3. Switch to the model you want

Change only model:

You want Model ID
Unlimited on a flat plan, 1M context broadnet/deepseek-v4.1-flash
Tools and structured output, 1M context broadnet/glm-5.3
Arabic text and images broadnet/qwen3.8-27b
Low price, works on free credit broadnet/gemma-4-26b-a4b

DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason before they answer, and the reasoning counts toward max_tokens. Give them 1,000 tokens or more. Here is why a small limit returns an empty answer.

4. Stream the answer

stream = client.chat.completions.create(
    model="broadnet/deepseek-v4.1-flash",
    stream=True,
    stream_options={"include_usage": True},
    messages=[{"role": "user", "content": "Count from one to three."}],
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)

With include_usage, the token counts arrive in the last chunk, which has no choices. The streaming guide covers dropped connections.

5. Keep the request ID

Every response carries an X-Request-ID header. Log it: BroadNet support asks for it. Parsed responses expose it as response._request_id. To read it alongside other headers:

raw = client.chat.completions.with_raw_response.create(
    model="broadnet/gemma-4-26b-a4b",
    messages=[{"role": "user", "content": "Hi"}],
)
print(raw.headers.get("x-request-id"))
response = raw.parse()

Pay-as-you-go keys allow 30 requests a minute and 4 in flight. What each error code means and when to retry.

Troubleshooting

  • AuthenticationError (401). The key is wrong, expired, or out of pay-as-you-go allowance. Check that BROADNET_API_KEY is set in the process that runs your code.
  • NotFoundError (404). Check that base_url ends in /v1 and that you call chat.completions, not responses. Then check the model ID, with the broadnet/ prefix.
  • Paid model fails on a new account. The $5 starting credit covers BroadNet Gemma 4 VL 26B (broadnet/gemma-4-26b-a4b) and the two BroadNet Transcribe models. Top up to use the other chat models: DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B.
  • Empty content. Reasoning counts toward max_tokens. Raise it to 1,000 or more. Why answers come back empty.
  • RateLimitError (429). You went over the per-minute or concurrent limit. The SDK retries twice by default; set max_retries on the client to change that.
  • Custom HTTP client errors after upgrading. The 3.x SDK uses HTTPX2. If you pass your own http_client, follow the SDK’s HTTPX2 migration guide.

Questions

Do I need a different SDK?

No. BroadNet Inference speaks the OpenAI chat-completions and audio-transcriptions shapes, so the official openai package works with a different base_url.

Can I keep using OPENAI_API_KEY?

You can, but a separate BROADNET_API_KEY variable avoids sending one provider's key to another by mistake.

Does the SDK read OPENAI_BASE_URL?

Yes. If you don't pass base_url, the client uses OPENAI_BASE_URL. Set it to https://inference.broadnet.ai/v1 and set OPENAI_API_KEY to your BroadNet key, and OpenAI() needs no arguments.

Can I use client.responses.create with BroadNet?

No. The SDK's README leads with the Responses API, but BroadNet's docs list only chat completions and audio transcriptions. Use client.chat.completions.create.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity