Moving an app from OpenAI to an OpenAI-compatible provider is a small code change if you use Chat Completions, and a rewrite if you use the Responses API. This post shows the exact diffs in Python, Node and cURL, what to do about Responses, tools, structured output and embeddings, and a smoke test to run before you move any traffic. Examples use BroadNet Inference at https://inference.broadnet.ai/v1; the same steps apply to any provider.
This is the code companion to the migration checklist, which covers rate limits, errors and cutover planning. For other providers’ base URLs, see OpenAI-compatible base URLs, checked.
What changes in the client setup?
Three values: base URL, API key and model ID.
Python
import os
from openai import OpenAI
# before: client = OpenAI() # reads OPENAI_API_KEY, calls api.openai.com
client = OpenAI(
base_url="https://inference.broadnet.ai/v1",
api_key=os.environ["BROADNET_API_KEY"],
)
r = client.chat.completions.create(
model="broadnet/gemma-4-26b-a4b", # was e.g. "gpt-..."
messages=[{"role": "user", "content": "Reply with one short sentence."}],
)
print(r.choices[0].message.content)
Node
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://inference.broadnet.ai/v1",
apiKey: process.env.BROADNET_API_KEY,
});
const r = await client.chat.completions.create({
model: "broadnet/gemma-4-26b-a4b",
messages: [{ role: "user", content: "Reply with one short sentence." }],
});
console.log(r.choices[0].message.content);
cURL
curl https://inference.broadnet.ai/v1/chat/completions \
-H "Authorization: Bearer $BROADNET_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"broadnet/gemma-4-26b-a4b","messages":[{"role":"user","content":"Reply with one short sentence."}]}'
Note the path: the base URL already ends in /v1, so the SDK adds /chat/completions. Doubling /v1 is the most common first error.
Can I switch without touching code?
Yes, for Chat Completions code. The Python SDK reads OPENAI_BASE_URL when no base_url is passed, and the Node client defaults baseURL to process.env['OPENAI_BASE_URL']. Both read OPENAI_API_KEY. Setting two environment variables moves every call.
That is also the risk. Every call moves, including embeddings, moderation, file uploads and anything else the new provider may not have. Use the environment switch only when your app makes chat calls and nothing else.
What if my code uses the Responses API?
OpenAI’s SDK READMEs now call the Responses API the primary API; the Python README calls Chat Completions “the previous standard (supported indefinitely)”. Many compatible providers implement only Chat Completions. BroadNet documents POST /v1/chat/completions and POST /v1/audio/transcriptions, so Responses calls need rewriting. The mapping:
| Responses API | Chat Completions |
|---|---|
client.responses.create(...) |
client.chat.completions.create(...) |
instructions="..." |
First message {"role": "system", "content": "..."} |
input="..." or a list of items |
messages=[...] |
response.output_text |
response.choices[0].message.content |
max_output_tokens |
max_tokens (BroadNet’s docs) or max_completion_tokens |
text={"format": {...}} |
response_format={...} |
previous_response_id, conversations |
Resend the full message history yourself |
Tool {"type": "function", "name": ...} |
Tool {"type": "function", "function": {"name": ...}} |
Output item function_call, reply with function_call_output and call_id |
message.tool_calls, reply with role tool and tool_call_id |
| Built-in tools (web search, file search) | Not available; run your own tool |
The before and after for a simple call:
# Before: Responses API
r = client.responses.create(model="gpt-...", instructions="Be brief.", input="What is RAG?")
text = r.output_text
# After: Chat Completions
r = client.chat.completions.create(
model="broadnet/gemma-4-26b-a4b",
messages=[{"role": "system", "content": "Be brief."},
{"role": "user", "content": "What is RAG?"}],
)
text = r.choices[0].message.content
The biggest structural change is state. Responses can keep conversation state on OpenAI’s side with previous_response_id. Chat Completions has no server-side state, so your code stores the message list and sends all of it each turn. Budget for that: the whole history is input tokens on every call.
How do tool calls change?
In Chat Completions, a tool definition nests the function under a function key, and the reply goes back as a tool message:
import json
tools = [{
"type": "function",
"function": {
"name": "get_order",
"description": "Look up an order by ID.",
"parameters": {"type": "object",
"properties": {"order_id": {"type": "string"}},
"required": ["order_id"]},
},
}]
messages = [{"role": "user", "content": "Where is order A-1042?"}]
r = client.chat.completions.create(model="broadnet/glm-5.3", messages=messages, tools=tools)
msg = r.choices[0].message
if msg.tool_calls:
messages.append(msg) # keep the assistant turn
for call in msg.tool_calls:
args = json.loads(call.function.arguments) # a JSON string: parse and validate it
messages.append({"role": "tool", "tool_call_id": call.id,
"content": json.dumps(lookup_order(args["order_id"]))})
r = client.chat.completions.create(model="broadnet/glm-5.3", messages=messages, tools=tools)
The model matters more than the shape. BroadNet’s docs list tools for GLM 5.3 and Qwen3.8 27B only. Don’t point a tool loop at a model whose docs don’t list tools, and validate arguments before you run anything. Tool calling with open models has a full validating loop.
What about structured output?
client.chat.completions.parse(..., response_format=MyPydanticModel) and Node’s zodResponseFormat both send response_format with a JSON schema. They work only if the provider accepts that shape on that model. BroadNet lists structured output for GLM 5.3 and Qwen3.8 27B but doesn’t name the request parameter. Run the four-step test in JSON mode vs json_schema once per model, and keep validating the parsed result whatever the provider promises.
What happens to embeddings?
If you change the base URL on a shared client and the provider has no embeddings endpoint, every client.embeddings.create call fails. BroadNet does not sell embeddings. Keep two clients:
chat = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])
embed = OpenAI(api_key=os.environ["OPENAI_API_KEY"]) # or any embeddings provider
vec = embed.embeddings.create(model="text-embedding-3-small", input=query).data[0].embedding
answer = chat.chat.completions.create(model="broadnet/glm-5.3", messages=build_prompt(query, search(vec)))
Never mix vectors from two embedding models in one index. If you change embedding provider too, re-embed the whole corpus. The embedding cost calculator prices that.
The same applies to anything else that isn’t chat: image generation, text to speech, moderation, the Batch API and file storage. BroadNet offers none of them. Speech to text does move: client.audio.transcriptions.create works against /v1/audio/transcriptions with WAV or MP3 and a language field (Arabic speech-to-text guide).
How does streaming code change?
The Responses API streams typed events, such as response.output_text.delta and response.completed. Chat Completions streams chunks, and the text sits in choices[0].delta.content. Usage is not sent unless you ask for it, and then it arrives in a final chunk with an empty choices list.
// Before (Responses): for await (const e of stream) if (e.type === "response.output_text.delta") out(e.delta);
const stream = await client.chat.completions.create({
model: "broadnet/gemma-4-26b-a4b",
messages: [{ role: "user", content: "Count from one to three." }],
stream: true,
stream_options: { include_usage: true },
});
for await (const chunk of stream) {
const text = chunk.choices[0]?.delta?.content;
if (text) process.stdout.write(text);
if (chunk.usage) console.log("\ntokens:", chunk.usage.total_tokens); // last chunk
}
Two habits keep stream code safe after the switch. Guard choices[0], because the usage chunk has no choices. And treat a stream that ends before the final data: [DONE] line as incomplete; BroadNet’s docs say to record the X-Request-ID and decide whether a new request is safe. Streaming chat completions has the full pattern in Python.
Do error handling and retries change?
The SDK’s error classes stay the same: AuthenticationError for 401, RateLimitError for 429, APIStatusError for the rest. What changes is what each status means at the new provider. On BroadNet, a 401 can mean an exhausted pay-as-you-go allowance, not only a wrong key, and 402 means insufficient balance for a purchase. Neither clears with a retry. A 429 can come from the rate, concurrency, token-rate or spend limit. Retry 429 and 5xx with backoff, and fix everything else. The errors and retries guide has the full table.
The SDK retries connection errors, 408, 409, 429 and 5xx twice by default. If you add your own retry loop, set max_retries=0 on the client so the two don’t stack.
Which parameters behave differently?
| Parameter | What to check |
|---|---|
max_tokens / max_completion_tokens |
OpenAI deprecates max_tokens; BroadNet’s docs use it. On reasoning models it includes thinking, so small values return empty answers (why) |
reasoning_effort |
BroadNet documents low; test other values before relying on them |
stream_options: {"include_usage": true} |
Documented by BroadNet; usage arrives in the last chunk |
n, logprobs, seed, logit_bias |
Not in BroadNet’s docs; test, or remove |
store, metadata |
OpenAI storage features; drop them |
image_url content parts |
Only on models listed with images: DeepSeek V4.1 Flash, Qwen3.8 27B, Gemma 4 VL 26B |
How do you test the switch?
Run a short smoke test against the new provider before any traffic moves. This one checks plain chat, streaming usage, the reasoning budget and authentication. Add a tool-call case if you use tools.
import os, openai
from openai import OpenAI
client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"], max_retries=0)
FAST, REASONING = "broadnet/gemma-4-26b-a4b", "broadnet/qwen3.8-27b"
msg = [{"role": "user", "content": "Reply with the word ok."}]
# 1. Plain chat returns text and usage
r = client.chat.completions.create(model=FAST, messages=msg)
assert r.choices[0].message.content and r.usage.prompt_tokens > 0
# 2. Streaming reports usage in the last chunk
chunks = list(client.chat.completions.create(model=FAST, messages=msg, stream=True,
stream_options={"include_usage": True}))
assert chunks[-1].usage is not None
# 3. A reasoning model needs room: a tiny budget can come back empty
r = client.chat.completions.create(model=REASONING, messages=msg, max_tokens=2000)
assert r.choices[0].finish_reason == "stop" and r.choices[0].message.content
# 4. A bad key raises AuthenticationError (401), which you should not retry
bad = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key="invalid", max_retries=0)
try:
bad.chat.completions.create(model=FAST, messages=msg)
raise AssertionError("expected 401")
except openai.AuthenticationError:
pass
print("smoke test passed")
Then compare quality, not just shape. Replay a sample of real prompts on both providers and score the answers; evaluate an LLM before switching shows how. The OpenAI to BroadNet migration tool helps you rewrite a snippet and lists which features it uses.
What breaks most often?
- A global base URL that also moves embeddings and file calls.
- Responses-only helpers such as
output_textleft in code that now calls Chat Completions. - Model IDs.
gpt-...names don’t exist elsewhere. On BroadNet they look likebroadnet/glm-5.3. - Empty replies from reasoning models with a small
max_tokens. - Rate limits. A BroadNet pay-as-you-go key allows 30 requests a minute and 4 in flight. Bulk jobs need a queue (429 handling).
Where BroadNet Inference fits
BroadNet Inference suits apps whose OpenAI usage is chat and transcription: assistants, extraction, summarisation, translation and Arabic speech. It also suits apps that want flat-rate DeepSeek V4.1 Flash plans for agents. It is the wrong target if you depend on the Responses API’s server-side state, built-in tools, embeddings, image generation, text to speech, realtime voice or the Batch API. Move those to providers that offer them, or keep them on OpenAI and move only the chat calls. Gemma 4 VL 26B and both transcription models run on the $5 starting credit; GLM 5.3, Qwen3.8 27B and DeepSeek V4.1 Flash need a top-up first.
