---
title: "LiteLLM OpenAI-compatible provider with api_base | BroadRouter"
description: "Call BroadNet Inference through LiteLLM with the openai/ model prefix and api_base, in the Python SDK or the proxy config.yaml, with matching rate limits."
url: https://broadrouter.com/guides/litellm-proxy-custom-provider/
lang: en
updated: 2026-11-16
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Route LiteLLM to BroadNet Inference (SDK and proxy)

Updated 2026-11-16

In LiteLLM, prefix the BroadNet model ID with openai/, as in openai/broadnet/glm-5.3, and set api\_base to https://inference.broadnet.ai/v1 with your BroadNet key as api\_key. In the proxy, put the same values under litellm\_params in config.yaml and give the model a short model\_name.

LiteLLM gives one interface to many LLM providers, as a Python SDK and as a proxy server. To reach an OpenAI-compatible API, its docs say to do two things: put `openai/` in front of the model name, and set `api_base`. BroadNet Inference uses that route.

## 1\. Python SDK

```bash
pip install litellm
export BROADNET_API_KEY='your-broadnet-key'
```

```python
import os
import litellm

response = litellm.completion(
    model="openai/broadnet/glm-5.3",
    api_base="https://inference.broadnet.ai/v1",
    api_key=os.environ["BROADNET_API_KEY"],
    max_tokens=4000,  # reasoning counts toward this
    messages=[{"role": "user", "content": "Name three uses of a bloom filter."}],
)
print(response.choices[0].message.content)
```

`openai/` selects the route; LiteLLM sends `broadnet/glm-5.3` to BroadNet. Don’t add anything to the base URL; LiteLLM’s OpenAI client adds the endpoint path.

LiteLLM’s docs note that the `openai` route uses the official OpenAI Python library, which needs an API key on every request. Pass `api_key` explicitly, as above, so LiteLLM never falls back to an `OPENAI_API_KEY` meant for OpenAI itself. The response comes back in the OpenAI shape, so the rest of your code doesn’t change.

## When to use the proxy

The SDK is enough inside one Python app. The proxy helps when several apps or languages need the same models: they all call one OpenAI-style endpoint with short model names, and the proxy holds the BroadNet key, applies rate limits and logs usage. Your apps then need no BroadNet-specific settings at all.

## 2\. Proxy server

Install the proxy extras and write `config.yaml`:

```bash
pip install 'litellm[proxy]'
```

```yaml
model_list:
  - model_name: glm
    litellm_params:
      model: openai/broadnet/glm-5.3
      api_base: https://inference.broadnet.ai/v1
      api_key: os.environ/BROADNET_API_KEY_GLM
      rpm: 30
  - model_name: deepseek-flash
    litellm_params:
      model: openai/broadnet/deepseek-v4.1-flash
      api_base: https://inference.broadnet.ai/v1
      api_key: os.environ/BROADNET_API_KEY_DEEPSEEK
      rpm: 30
  - model_name: qwen
    litellm_params:
      model: openai/broadnet/qwen3.8-27b
      api_base: https://inference.broadnet.ai/v1
      api_key: os.environ/BROADNET_API_KEY_QWEN
      rpm: 30

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
```

Start it and call it like any OpenAI endpoint:

```bash
litellm --config config.yaml

# in a second terminal
curl http://localhost:4000/chat/completions \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "glm", "messages": [{"role": "user", "content": "Hello"}]}'
```

`os.environ/BROADNET_API_KEY_GLM` tells LiteLLM to read that key from the environment. Each entry uses its own BroadNet pay-as-you-go key, so `rpm: 30`, LiteLLM’s optional per-deployment rate limit, matches each key’s limit. If several entries share one key, they share that one limit, so split the number between them. `master_key` makes the proxy require a key on every call; LiteLLM’s docs say it must start with `sk-`. Export `LITELLM_MASTER_KEY` before you start the proxy.

## Which model to pick

| model\_name | BroadNet model | Use |
| --- | --- | --- |
| `glm` | [GLM 5.3](https://broadrouter.com/models/glm-5.3/) | Tools, structured output, 1M context |
| `deepseek-flash` | [DeepSeek V4.1 Flash](https://broadrouter.com/models/deepseek-v4.1-flash/) | Long context; the unlimited-plan model |
| `qwen` | [Qwen3.8 27B](https://broadrouter.com/models/qwen3.8-27b/) | Images, tools, Arabic |
| (add if needed) | [Gemma 4 VL 26B](https://broadrouter.com/models/gemma-4-vl-26b/) | Low per-token price; works on the $5 credit |

Prices are on the [pricing page](https://broadrouter.com/pricing/). If you are moving from another gateway, the [migration checklist](https://broadrouter.com/blog/openai-compatible-api-migration-checklist/) covers what to test.

## Troubleshooting

-   **401.** The BroadNet key is wrong, expired, or out of pay-as-you-go allowance. If the 401 comes from the proxy itself, check the LiteLLM key you sent instead.
-   **Not Found.** Make sure `api_base` is `https://inference.broadnet.ai/v1` with `/v1`. If the base URL is right, check the model ID after `openai/` matches BroadNet’s exactly.
-   **Model refused on a new account.** GLM 5.3, Qwen3.8 27B and DeepSeek V4.1 Flash need a top-up first; the $5 credit covers only `broadnet/gemma-4-26b-a4b` and the two Transcribe models.
-   **Empty content.** Reasoning counts toward `max_tokens`. Raise it or send `reasoning_effort="low"`. [Why answers come back empty](https://broadrouter.com/guides/reasoning-tokens-empty-response/).
-   **429.** A pay-as-you-go key allows 30 requests a minute and 4 at once. LiteLLM’s `rpm` helps with the first; keep concurrent callers at 4 or fewer per key. See [errors and retries](https://broadrouter.com/guides/errors-and-retries/).
-   **Parallel agents on a plan key.** One [plan session](https://broadrouter.com/guides/unlimited-llm-api-sessions/) runs one request at a time. Running several agents through the proxy on one session makes them queue; buy one session per parallel agent.

## Terms in this guide

[Pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/)

Pay-as-you-go is a pricing model in which you pay only for what you use, metered per unit such as tokens, requests or audio minutes, with no subscription or minimum commitment. Most LLM APIs bill this way, from a prepaid balance or a periodic invoice.

[Rate limit](https://broadrouter.com/glossary/rate-limit/)

A rate limit is a cap an API places on how much a client can use it in a time window, such as requests per minute or tokens per minute. Requests over the limit are refused, usually with HTTP 429 Too Many Requests, until usage falls back under it.

[Base URL](https://broadrouter.com/glossary/base-url/)

A base URL is the fixed root address of an API, such as https://api.example.com/v1, that a client puts in front of every endpoint path, such as /chat/completions. In OpenAI-compatible SDKs, changing the base URL, together with the API key and model ID, points the same code at a different provider.

[Concurrency](https://broadrouter.com/glossary/concurrency/)

Concurrency is the number of requests a client has in flight at the same time. LLM APIs often cap it per key or plan, separately from requests per minute, because each open request holds GPU capacity until its response has finished.

[Tool calling](https://broadrouter.com/glossary/tool-calling/)

Tool calling, also called function calling, is an LLM API feature in which the model returns a structured request to run a function the developer has defined, with JSON arguments. The application runs the function and sends the result back; the model never runs code itself.

[API key](https://broadrouter.com/glossary/api-key/)

An API key is a secret string that identifies and authorizes a client calling an API. LLM APIs usually expect it in an Authorization: Bearer header, and anyone who holds the key can make requests that are billed to the account that owns it.

[OpenAI-compatible API](https://broadrouter.com/glossary/openai-compatible-api/)

An OpenAI-compatible API is an HTTP API that accepts the same request and response formats as the OpenAI API, such as POST /v1/chat/completions, so OpenAI SDKs and tools can use it by changing only the base URL, the API key and the model name.

[Prepaid credit](https://broadrouter.com/glossary/prepaid-credit/)

Prepaid credit is money added to an API account before use. Each request deducts its cost from the balance, and when the balance runs out, metered requests stop until you top up. It caps spend at what you have paid in.

## Questions

### Why does LiteLLM return a Not Found error?

LiteLLM's docs say to make sure api\_base ends in /v1. Use https://inference.broadnet.ai/v1 and don't add /chat/completions.

### Why is the model name openai/broadnet/glm-5.3?

The openai/ prefix tells LiteLLM to call an OpenAI-style /chat/completions endpoint. LiteLLM uses the prefix to pick the route and sends the rest, broadnet/glm-5.3, which is the BroadNet model ID.

### Can I put a BroadNet unlimited plan key behind a shared LiteLLM proxy?

Plan keys can't be shared or resold, and one session runs one request at a time. Use them for your own agents. For a team proxy, use pay-as-you-go keys.

### Does LiteLLM need OPENAI\_API\_KEY to call BroadNet?

The openai route needs a key on every request, from api\_key or OPENAI\_API\_KEY. Pass your BroadNet key as api\_key, or os.environ/BROADNET\_API\_KEY in the proxy, so LiteLLM doesn't send a real OpenAI key to BroadNet.

## Related guides

-   [Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/)
-   [BroadNet Inference error codes and when to retry](https://broadrouter.com/guides/errors-and-retries/)
-   [Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/)
-   [Use LangChain with BroadNet Inference](https://broadrouter.com/guides/langchain-openai-base-url/)
-   [All tools that work with BroadNet Inference](https://broadrouter.com/integrations/): SDKs, coding agents, chat UIs and automation tools, with exact settings for each

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=guides-litellm-proxy-custom-provider&utm_content=page-cta) ·

**Sources**

-   [LiteLLM docs, OpenAI-compatible endpoints (checked 2026-11-14)](https://docs.litellm.ai/docs/providers/openai_compatible)
-   [LiteLLM docs, proxy config.yaml (checked 2026-11-14)](https://docs.litellm.ai/docs/proxy/configs)
-   [BroadNet Inference API reference](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=guides-litellm-proxy-custom-provider&utm_content=sources)
