BroadRouter
Get started

Route LiteLLM to BroadNet Inference (SDK and proxy)

Updated

In LiteLLM, prefix the BroadNet model ID with openai/, as in openai/broadnet/glm-5.3, and set api_base to https://inference.broadnet.ai/v1 with your BroadNet key as api_key. In the proxy, put the same values under litellm_params in config.yaml and give the model a short model_name.

LiteLLM gives one interface to many LLM providers, as a Python SDK and as a proxy server. To reach an OpenAI-compatible API, its docs say to do two things: put openai/ in front of the model name, and set api_base. BroadNet Inference uses that route.

1. Python SDK

pip install litellm
export BROADNET_API_KEY='your-broadnet-key'
import os
import litellm

response = litellm.completion(
    model="openai/broadnet/glm-5.3",
    api_base="https://inference.broadnet.ai/v1",
    api_key=os.environ["BROADNET_API_KEY"],
    max_tokens=4000,  # reasoning counts toward this
    messages=[{"role": "user", "content": "Name three uses of a bloom filter."}],
)
print(response.choices[0].message.content)

openai/ selects the route; LiteLLM sends broadnet/glm-5.3 to BroadNet. Don’t add anything to the base URL; LiteLLM’s OpenAI client adds the endpoint path.

LiteLLM’s docs note that the openai route uses the official OpenAI Python library, which needs an API key on every request. Pass api_key explicitly, as above, so LiteLLM never falls back to an OPENAI_API_KEY meant for OpenAI itself. The response comes back in the OpenAI shape, so the rest of your code doesn’t change.

When to use the proxy

The SDK is enough inside one Python app. The proxy helps when several apps or languages need the same models: they all call one OpenAI-style endpoint with short model names, and the proxy holds the BroadNet key, applies rate limits and logs usage. Your apps then need no BroadNet-specific settings at all.

2. Proxy server

Install the proxy extras and write config.yaml:

pip install 'litellm[proxy]'
model_list:
  - model_name: glm
    litellm_params:
      model: openai/broadnet/glm-5.3
      api_base: https://inference.broadnet.ai/v1
      api_key: os.environ/BROADNET_API_KEY_GLM
      rpm: 30
  - model_name: deepseek-flash
    litellm_params:
      model: openai/broadnet/deepseek-v4.1-flash
      api_base: https://inference.broadnet.ai/v1
      api_key: os.environ/BROADNET_API_KEY_DEEPSEEK
      rpm: 30
  - model_name: qwen
    litellm_params:
      model: openai/broadnet/qwen3.8-27b
      api_base: https://inference.broadnet.ai/v1
      api_key: os.environ/BROADNET_API_KEY_QWEN
      rpm: 30

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

Start it and call it like any OpenAI endpoint:

litellm --config config.yaml

# in a second terminal
curl http://localhost:4000/chat/completions \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "glm", "messages": [{"role": "user", "content": "Hello"}]}'

os.environ/BROADNET_API_KEY_GLM tells LiteLLM to read that key from the environment. Each entry uses its own BroadNet pay-as-you-go key, so rpm: 30, LiteLLM’s optional per-deployment rate limit, matches each key’s limit. If several entries share one key, they share that one limit, so split the number between them. master_key makes the proxy require a key on every call; LiteLLM’s docs say it must start with sk-. Export LITELLM_MASTER_KEY before you start the proxy.

Which model to pick

model_name BroadNet model Use
glm GLM 5.3 Tools, structured output, 1M context
deepseek-flash DeepSeek V4.1 Flash Long context; the unlimited-plan model
qwen Qwen3.8 27B Images, tools, Arabic
(add if needed) Gemma 4 VL 26B Low per-token price; works on the $5 credit

Prices are on the pricing page. If you are moving from another gateway, the migration checklist covers what to test.

Troubleshooting

  • 401. The BroadNet key is wrong, expired, or out of pay-as-you-go allowance. If the 401 comes from the proxy itself, check the LiteLLM key you sent instead.
  • Not Found. Make sure api_base is https://inference.broadnet.ai/v1 with /v1. If the base URL is right, check the model ID after openai/ matches BroadNet’s exactly.
  • Model refused on a new account. GLM 5.3, Qwen3.8 27B and DeepSeek V4.1 Flash need a top-up first; the $5 credit covers only broadnet/gemma-4-26b-a4b and the two Transcribe models.
  • Empty content. Reasoning counts toward max_tokens. Raise it or send reasoning_effort="low". Why answers come back empty.
  • 429. A pay-as-you-go key allows 30 requests a minute and 4 at once. LiteLLM’s rpm helps with the first; keep concurrent callers at 4 or fewer per key. See errors and retries.
  • Parallel agents on a plan key. One plan session runs one request at a time. Running several agents through the proxy on one session makes them queue; buy one session per parallel agent.

Questions

Why does LiteLLM return a Not Found error?

LiteLLM's docs say to make sure api_base ends in /v1. Use https://inference.broadnet.ai/v1 and don't add /chat/completions.

Why is the model name openai/broadnet/glm-5.3?

The openai/ prefix tells LiteLLM to call an OpenAI-style /chat/completions endpoint. LiteLLM uses the prefix to pick the route and sends the rest, broadnet/glm-5.3, which is the BroadNet model ID.

Can I put a BroadNet unlimited plan key behind a shared LiteLLM proxy?

Plan keys can't be shared or resold, and one session runs one request at a time. Use them for your own agents. For a team proxy, use pay-as-you-go keys.

Does LiteLLM need OPENAI_API_KEY to call BroadNet?

The openai route needs a key on every request, from api_key or OPENAI_API_KEY. Pass your BroadNet key as api_key, or os.environ/BROADNET_API_KEY in the proxy, so LiteLLM doesn't send a real OpenAI key to BroadNet.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity