BroadRouter
Get started

Switching to an OpenAI-compatible API: a migration checklist

By the BroadNet Inference team · 5 min read · Published · Updated

To move to an OpenAI-compatible API, change three settings (base_url, api_key and model) and then check what differs: model IDs, how reasoning tokens count toward max_tokens, how streaming reports usage, which errors to retry, rate limits and key security. For BroadNet Inference the base URL is https://inference.broadnet.ai/v1.

Most LLM providers now speak the OpenAI API format. That makes switching look like a one-line change, and to get a first answer, it is. To run production traffic, work through this list. Every item comes from a real way that migrations break.

0. What does your app use today?

Before you change anything, write down what your app depends on. Search the code for every call to the SDK and list it. If you call OpenAI’s Whisper API, note that OpenAI lists whisper-1 as shutting down on February 26, 2027; Whisper API alternatives covers the audio side.

Feature Where to look Test after switching
Chat completions Every chat.completions.create Same prompts, compare answers
Streaming stream=True Chunks arrive, ends with [DONE], usage present
Tool calling tools= Tool calls parse and run
Structured output response_format= JSON validates against your schema
Images Image parts in messages Model accepts them
Audio transcription audio.transcriptions Text and language correct
Embeddings, files, batches Other endpoints Check the provider offers them at all

The last row matters. “OpenAI-compatible” usually covers chat completions first. BroadNet Inference’s catalogue lists chat models on /v1/chat/completions and speech-to-text models on /v1/audio/transcriptions; if you use other endpoints, keep them on your current provider or find a replacement.

1. Which three settings change?

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://inference.broadnet.ai/v1",
    api_key=os.environ["BROADNET_API_KEY"],
)

The name of the base URL setting differs by library: base_url in the Python SDK and LangChain’s ChatOpenAI, baseURL in the Node.js SDK. The Python SDK also reads the OPENAI_BASE_URL environment variable, according to its README. Check for a stale value in your environment, or requests can go to the old provider without an error.

Keep API keys in environment variables or a secret manager, never in code. Setup for each SDK: Python, Node.js, LangChain. To build and test a first request by hand, use the API request builder. To apply the base URL, key and model edits to existing code, paste it into the OpenAI to BroadNet migration helper, which also lists what to check.

2. What are the new model IDs?

Every provider names models its own way. On BroadNet Inference:

Model ID
DeepSeek V4.1 Flash broadnet/deepseek-v4.1-flash
GLM 5.3 broadnet/glm-5.3
Qwen3.8 27B broadnet/qwen3.8-27b
Gemma 4 VL 26B broadnet/gemma-4-26b-a4b

Put the model ID in configuration, not code, so switching later is a deploy, not a rewrite. Each model’s limits and prices are on its page: DeepSeek V4.1 Flash, GLM 5.3, Qwen3.8 27B and Gemma 4 VL 26B.

3. Why do answers come back empty after switching?

This is the most common surprise. DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason before they answer, and the reasoning tokens count toward max_tokens. A limit of 256 that worked on a non-reasoning model can be spent entirely on thinking, so the answer comes back empty. Leave 1,000 tokens or more, or set reasoning_effort to low for simple tasks. Full explanation.

4. Does streaming still report usage?

Streams use Server-Sent Events and end with data: [DONE]. Token usage arrives in the last chunk only if you ask:

stream = client.chat.completions.create(
    model="broadnet/deepseek-v4.1-flash", messages=messages,
    stream=True, stream_options={"include_usage": True},
)

Without it, your cost dashboard reads zero. If a stream ends before [DONE], treat it as incomplete. Streaming guide.

5. Which errors should you retry?

Status Meaning Action
401 Bad or revoked key, or exhausted pay-as-you-go allowance Fix, don’t retry
402 Balance or budget Top up, don’t retry
403 Refused by model, route or IP policy Fix the mismatch
413 Request too large Shrink, don’t retry
429 Rate, concurrency or spend limit Back off and retry
5xx Server side Retry with bounded backoff

The official Python SDK already retries some errors: its README says connection errors, 408, 409, 429 and 5xx responses are retried twice by default with a short exponential backoff. Set max_retries explicitly so the behaviour is a decision, not a default. Log the request ID from every response so support can trace a problem. All codes.

6. What are the new rate limits?

Know the new limits before traffic arrives. A BroadNet pay-as-you-go key allows 30 requests a minute and 4 in flight. Unlimited plans run one request at a time per session. Size your worker pool to match, or you will see 429s on day one; the rate limit calculator does the sizing. Sizing for parallel agents.

7. Will your prompts still fit?

Check context windows: 1,048,576 tokens on DeepSeek V4.1 Flash and GLM 5.3, 262,144 on Qwen3.8 27B, 32,768 on Gemma 4 VL 26B. A prompt that fit your old model may not fit a smaller window; check with the context window calculator. Keep stable prompt content first to benefit from cached input pricing.

8. What about keys and data retention?

Lock production keys to your servers’ IPs with an IP allow-list. Ask the new provider what it retains; see the ZDR checklist.

9. How do you cut over safely?

  1. Send a copy of a day’s real requests to both providers.
  2. Compare answers on a sample, plus latency, errors and cost per request.
  3. Move 10% of traffic, watch for a day, then move the rest.
  4. Keep the old provider configured as a fallback for a week.

If you run many services, a proxy such as LiteLLM lets you switch providers in one place rather than in every app.

What are the common migration mistakes?

  • Missing /v1 in the base URL. Requests return 404. Use the full URL.
  • Old model names in config. The new provider doesn’t know them.
  • Provider-specific fields left in requests. Fields that only the old provider understood may be rejected or ignored. Remove them from generic clients.
  • Testing only one prompt. Run the inventory from step 0, feature by feature.
  • Cutting over on a Friday. Leave time to watch a full day of real traffic.

Where BroadNet Inference fits

BroadNet Inference is a drop-in target for apps that use chat completions and audio transcription through the OpenAI SDKs: change three settings and work through the list above. Its catalogue lists chat and speech-to-text models only, so if you depend on other endpoints, such as embeddings or file storage, keep those with another provider. New accounts start with $5 of credit. Of the chat models, only Gemma 4 VL 26B runs on it (it also covers the two Transcribe models), so a shadow test on DeepSeek, GLM or Qwen needs a top-up.

Questions

Do I need a different SDK?

No. The official OpenAI SDKs for Python and Node.js accept a custom base URL. Frameworks built on them, such as LangChain's ChatOpenAI, accept it too.

How do I change the base URL in the OpenAI SDK?

Pass base_url to the client in Python or baseURL in Node.js, or set the OPENAI_BASE_URL environment variable. For BroadNet Inference the value is https://inference.broadnet.ai/v1, and the key goes in api_key.

Why do I get an empty response after switching?

Reasoning models such as DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B think before they answer, and the thinking counts toward max_tokens. Raise max_tokens to 1,000 or more, or set reasoning_effort to low.

Why do I get a 404 or model not found error after switching?

Usually the model ID is still the old provider's name, or the base URL is missing its /v1 path. Use the new provider's exact model ID and the full base URL.

Which errors should I retry?

Retry 429 and 5xx with bounded exponential backoff. Fix 401, 402 and 413 before retrying. The errors guide on this site lists every code.

Is OpenAI-compatible the same as identical?

No. The request and response shapes match, but model IDs, supported parameters, limits, usage fields and error details can differ. Test every feature your app uses.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity