BroadRouter
Get started

Migrating off the OpenAI API: a code walkthrough in Python, Node, cURL

By the BroadNet Inference team · 6 min read · Published

Change the client's base URL, API key and model ID. That is enough for Chat Completions calls. Code on the Responses API must be rewritten to Chat Completions: input becomes messages, output_text becomes choices[0].message.content, and tools lose a level of nesting. Keep embeddings on a separate client if the new provider has none, then run a smoke test before switching traffic.

Moving an app from OpenAI to an OpenAI-compatible provider is a small code change if you use Chat Completions, and a rewrite if you use the Responses API. This post shows the exact diffs in Python, Node and cURL, what to do about Responses, tools, structured output and embeddings, and a smoke test to run before you move any traffic. Examples use BroadNet Inference at https://inference.broadnet.ai/v1; the same steps apply to any provider.

This is the code companion to the migration checklist, which covers rate limits, errors and cutover planning. For other providers’ base URLs, see OpenAI-compatible base URLs, checked.

What changes in the client setup?

Three values: base URL, API key and model ID.

Python

import os
from openai import OpenAI

# before: client = OpenAI()  # reads OPENAI_API_KEY, calls api.openai.com
client = OpenAI(
    base_url="https://inference.broadnet.ai/v1",
    api_key=os.environ["BROADNET_API_KEY"],
)
r = client.chat.completions.create(
    model="broadnet/gemma-4-26b-a4b",          # was e.g. "gpt-..."
    messages=[{"role": "user", "content": "Reply with one short sentence."}],
)
print(r.choices[0].message.content)

Node

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://inference.broadnet.ai/v1",
  apiKey: process.env.BROADNET_API_KEY,
});
const r = await client.chat.completions.create({
  model: "broadnet/gemma-4-26b-a4b",
  messages: [{ role: "user", content: "Reply with one short sentence." }],
});
console.log(r.choices[0].message.content);

cURL

curl https://inference.broadnet.ai/v1/chat/completions \
  -H "Authorization: Bearer $BROADNET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"broadnet/gemma-4-26b-a4b","messages":[{"role":"user","content":"Reply with one short sentence."}]}'

Note the path: the base URL already ends in /v1, so the SDK adds /chat/completions. Doubling /v1 is the most common first error.

Can I switch without touching code?

Yes, for Chat Completions code. The Python SDK reads OPENAI_BASE_URL when no base_url is passed, and the Node client defaults baseURL to process.env['OPENAI_BASE_URL']. Both read OPENAI_API_KEY. Setting two environment variables moves every call.

That is also the risk. Every call moves, including embeddings, moderation, file uploads and anything else the new provider may not have. Use the environment switch only when your app makes chat calls and nothing else.

What if my code uses the Responses API?

OpenAI’s SDK READMEs now call the Responses API the primary API; the Python README calls Chat Completions “the previous standard (supported indefinitely)”. Many compatible providers implement only Chat Completions. BroadNet documents POST /v1/chat/completions and POST /v1/audio/transcriptions, so Responses calls need rewriting. The mapping:

Responses API Chat Completions
client.responses.create(...) client.chat.completions.create(...)
instructions="..." First message {"role": "system", "content": "..."}
input="..." or a list of items messages=[...]
response.output_text response.choices[0].message.content
max_output_tokens max_tokens (BroadNet’s docs) or max_completion_tokens
text={"format": {...}} response_format={...}
previous_response_id, conversations Resend the full message history yourself
Tool {"type": "function", "name": ...} Tool {"type": "function", "function": {"name": ...}}
Output item function_call, reply with function_call_output and call_id message.tool_calls, reply with role tool and tool_call_id
Built-in tools (web search, file search) Not available; run your own tool

The before and after for a simple call:

# Before: Responses API
r = client.responses.create(model="gpt-...", instructions="Be brief.", input="What is RAG?")
text = r.output_text

# After: Chat Completions
r = client.chat.completions.create(
    model="broadnet/gemma-4-26b-a4b",
    messages=[{"role": "system", "content": "Be brief."},
              {"role": "user", "content": "What is RAG?"}],
)
text = r.choices[0].message.content

The biggest structural change is state. Responses can keep conversation state on OpenAI’s side with previous_response_id. Chat Completions has no server-side state, so your code stores the message list and sends all of it each turn. Budget for that: the whole history is input tokens on every call.

How do tool calls change?

In Chat Completions, a tool definition nests the function under a function key, and the reply goes back as a tool message:

import json

tools = [{
    "type": "function",
    "function": {
        "name": "get_order",
        "description": "Look up an order by ID.",
        "parameters": {"type": "object",
                       "properties": {"order_id": {"type": "string"}},
                       "required": ["order_id"]},
    },
}]
messages = [{"role": "user", "content": "Where is order A-1042?"}]
r = client.chat.completions.create(model="broadnet/glm-5.3", messages=messages, tools=tools)
msg = r.choices[0].message
if msg.tool_calls:
    messages.append(msg)                                   # keep the assistant turn
    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)         # a JSON string: parse and validate it
        messages.append({"role": "tool", "tool_call_id": call.id,
                         "content": json.dumps(lookup_order(args["order_id"]))})
    r = client.chat.completions.create(model="broadnet/glm-5.3", messages=messages, tools=tools)

The model matters more than the shape. BroadNet’s docs list tools for GLM 5.3 and Qwen3.8 27B only. Don’t point a tool loop at a model whose docs don’t list tools, and validate arguments before you run anything. Tool calling with open models has a full validating loop.

What about structured output?

client.chat.completions.parse(..., response_format=MyPydanticModel) and Node’s zodResponseFormat both send response_format with a JSON schema. They work only if the provider accepts that shape on that model. BroadNet lists structured output for GLM 5.3 and Qwen3.8 27B but doesn’t name the request parameter. Run the four-step test in JSON mode vs json_schema once per model, and keep validating the parsed result whatever the provider promises.

What happens to embeddings?

If you change the base URL on a shared client and the provider has no embeddings endpoint, every client.embeddings.create call fails. BroadNet does not sell embeddings. Keep two clients:

chat = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])
embed = OpenAI(api_key=os.environ["OPENAI_API_KEY"])   # or any embeddings provider

vec = embed.embeddings.create(model="text-embedding-3-small", input=query).data[0].embedding
answer = chat.chat.completions.create(model="broadnet/glm-5.3", messages=build_prompt(query, search(vec)))

Never mix vectors from two embedding models in one index. If you change embedding provider too, re-embed the whole corpus. The embedding cost calculator prices that.

The same applies to anything else that isn’t chat: image generation, text to speech, moderation, the Batch API and file storage. BroadNet offers none of them. Speech to text does move: client.audio.transcriptions.create works against /v1/audio/transcriptions with WAV or MP3 and a language field (Arabic speech-to-text guide).

How does streaming code change?

The Responses API streams typed events, such as response.output_text.delta and response.completed. Chat Completions streams chunks, and the text sits in choices[0].delta.content. Usage is not sent unless you ask for it, and then it arrives in a final chunk with an empty choices list.

// Before (Responses): for await (const e of stream) if (e.type === "response.output_text.delta") out(e.delta);
const stream = await client.chat.completions.create({
  model: "broadnet/gemma-4-26b-a4b",
  messages: [{ role: "user", content: "Count from one to three." }],
  stream: true,
  stream_options: { include_usage: true },
});
for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content;
  if (text) process.stdout.write(text);
  if (chunk.usage) console.log("\ntokens:", chunk.usage.total_tokens);   // last chunk
}

Two habits keep stream code safe after the switch. Guard choices[0], because the usage chunk has no choices. And treat a stream that ends before the final data: [DONE] line as incomplete; BroadNet’s docs say to record the X-Request-ID and decide whether a new request is safe. Streaming chat completions has the full pattern in Python.

Do error handling and retries change?

The SDK’s error classes stay the same: AuthenticationError for 401, RateLimitError for 429, APIStatusError for the rest. What changes is what each status means at the new provider. On BroadNet, a 401 can mean an exhausted pay-as-you-go allowance, not only a wrong key, and 402 means insufficient balance for a purchase. Neither clears with a retry. A 429 can come from the rate, concurrency, token-rate or spend limit. Retry 429 and 5xx with backoff, and fix everything else. The errors and retries guide has the full table.

The SDK retries connection errors, 408, 409, 429 and 5xx twice by default. If you add your own retry loop, set max_retries=0 on the client so the two don’t stack.

Which parameters behave differently?

Parameter What to check
max_tokens / max_completion_tokens OpenAI deprecates max_tokens; BroadNet’s docs use it. On reasoning models it includes thinking, so small values return empty answers (why)
reasoning_effort BroadNet documents low; test other values before relying on them
stream_options: {"include_usage": true} Documented by BroadNet; usage arrives in the last chunk
n, logprobs, seed, logit_bias Not in BroadNet’s docs; test, or remove
store, metadata OpenAI storage features; drop them
image_url content parts Only on models listed with images: DeepSeek V4.1 Flash, Qwen3.8 27B, Gemma 4 VL 26B

How do you test the switch?

Run a short smoke test against the new provider before any traffic moves. This one checks plain chat, streaming usage, the reasoning budget and authentication. Add a tool-call case if you use tools.

import os, openai
from openai import OpenAI

client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"], max_retries=0)
FAST, REASONING = "broadnet/gemma-4-26b-a4b", "broadnet/qwen3.8-27b"
msg = [{"role": "user", "content": "Reply with the word ok."}]

# 1. Plain chat returns text and usage
r = client.chat.completions.create(model=FAST, messages=msg)
assert r.choices[0].message.content and r.usage.prompt_tokens > 0

# 2. Streaming reports usage in the last chunk
chunks = list(client.chat.completions.create(model=FAST, messages=msg, stream=True,
                                             stream_options={"include_usage": True}))
assert chunks[-1].usage is not None

# 3. A reasoning model needs room: a tiny budget can come back empty
r = client.chat.completions.create(model=REASONING, messages=msg, max_tokens=2000)
assert r.choices[0].finish_reason == "stop" and r.choices[0].message.content

# 4. A bad key raises AuthenticationError (401), which you should not retry
bad = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key="invalid", max_retries=0)
try:
    bad.chat.completions.create(model=FAST, messages=msg)
    raise AssertionError("expected 401")
except openai.AuthenticationError:
    pass
print("smoke test passed")

Then compare quality, not just shape. Replay a sample of real prompts on both providers and score the answers; evaluate an LLM before switching shows how. The OpenAI to BroadNet migration tool helps you rewrite a snippet and lists which features it uses.

What breaks most often?

  • A global base URL that also moves embeddings and file calls.
  • Responses-only helpers such as output_text left in code that now calls Chat Completions.
  • Model IDs. gpt-... names don’t exist elsewhere. On BroadNet they look like broadnet/glm-5.3.
  • Empty replies from reasoning models with a small max_tokens.
  • Rate limits. A BroadNet pay-as-you-go key allows 30 requests a minute and 4 in flight. Bulk jobs need a queue (429 handling).

Where BroadNet Inference fits

BroadNet Inference suits apps whose OpenAI usage is chat and transcription: assistants, extraction, summarisation, translation and Arabic speech. It also suits apps that want flat-rate DeepSeek V4.1 Flash plans for agents. It is the wrong target if you depend on the Responses API’s server-side state, built-in tools, embeddings, image generation, text to speech, realtime voice or the Batch API. Move those to providers that offer them, or keep them on OpenAI and move only the chat calls. Gemma 4 VL 26B and both transcription models run on the $5 starting credit; GLM 5.3, Qwen3.8 27B and DeepSeek V4.1 Flash need a top-up first.

Questions

Do I need a different SDK to use an OpenAI-compatible API?

No. Keep the official openai package and pass a different base_url (Python) or baseURL (Node), plus the new provider's key. The request and response objects stay the same for the endpoints the provider supports.

Does an OpenAI-compatible API support the Responses API?

Some do; many support only Chat Completions. Check the provider's endpoint list. BroadNet Inference documents /v1/chat/completions and /v1/audio/transcriptions, so Responses code must be rewritten to Chat Completions.

What happens to my embeddings when I switch?

They break if the new provider has no embeddings endpoint and you changed the base URL globally. BroadNet does not sell embeddings, so keep an embeddings client pointed at your embeddings provider. Never mix vectors from two embedding models in one index.

Should I send max_tokens or max_completion_tokens?

OpenAI marks max_tokens as deprecated in favour of max_completion_tokens. Many compatible providers document max_tokens instead, and BroadNet's docs use max_tokens. Send what the provider documents and confirm the limit takes effect in your smoke test.

Will OpenAI's structured output helpers still work?

Only if the provider accepts response_format with json_schema on the model you use. BroadNet lists structured output for GLM 5.3 and Qwen3.8 27B but does not name the request parameter, so test it, and keep validating the parsed result.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity