---
title: "Migrate from the OpenAI API - code diffs and tests | BroadRouter"
description: "The code changes to move from OpenAI to an OpenAI-compatible API: client setup, Responses to Chat Completions, tools, streaming, embeddings and a smoke test."
url: https://broadrouter.com/blog/migrate-from-openai-api-code-walkthrough/
lang: en
updated: 2026-11-15
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Migrating off the OpenAI API: a code walkthrough in Python, Node, cURL

By the BroadNet Inference team · 6 min read · Published 2026-11-15

Change the client's base URL, API key and model ID. That is enough for Chat Completions calls. Code on the Responses API must be rewritten to Chat Completions: input becomes messages, output\_text becomes choices\[0\].message.content, and tools lose a level of nesting. Keep embeddings on a separate client if the new provider has none, then run a smoke test before switching traffic.

## Key takeaways

-   For Chat Completions code, the switch is three values: base\_url, api\_key and model. Both official SDKs also read OPENAI\_BASE\_URL and OPENAI\_API\_KEY from the environment.
-   Setting OPENAI\_BASE\_URL moves every call, embeddings included. If the new provider has no embeddings endpoint, give embeddings their own client.
-   Responses API code needs a rewrite: input to messages, instructions to a system message, text.format to response\_format, and flat tool definitions to nested ones.
-   BroadNet Inference documents /v1/chat/completions, /v1/audio/transcriptions and GET /v1/models, and no embeddings, files or batch endpoint. Its docs list tools and structured output for GLM 5.3 and Qwen3.8 27B.
-   Test before you switch: plain chat, streaming usage, a reasoning budget and one bad key, plus a tool call if you use tools.

**On this page**

1.  What changes in the client setup?
2.  What if my code uses the Responses API?
3.  How do tool calls change?
4.  What about structured output?
5.  What happens to embeddings?
6.  How does streaming code change?
7.  Do error handling and retries change?
8.  Which parameters behave differently?
9.  How do you test the switch?
10.  What breaks most often?
11.  Where BroadNet Inference fits

Moving an app from OpenAI to an OpenAI-compatible provider is a small code change if you use Chat Completions, and a rewrite if you use the Responses API. This post shows the exact diffs in Python, Node and cURL, what to do about Responses, tools, structured output and embeddings, and a smoke test to run before you move any traffic. Examples use BroadNet Inference at `https://inference.broadnet.ai/v1`; the same steps apply to any provider.

This is the code companion to the [migration checklist](https://broadrouter.com/blog/openai-compatible-api-migration-checklist/), which covers rate limits, errors and cutover planning. For other providers’ base URLs, see [OpenAI-compatible base URLs, checked](https://broadrouter.com/blog/openai-compatible-api-base-urls/).

## What changes in the client setup?

Three values: base URL, API key and model ID.

**Python**

```python
import os
from openai import OpenAI

# before: client = OpenAI()  # reads OPENAI_API_KEY, calls api.openai.com
client = OpenAI(
    base_url="https://inference.broadnet.ai/v1",
    api_key=os.environ["BROADNET_API_KEY"],
)
r = client.chat.completions.create(
    model="broadnet/gemma-4-26b-a4b",          # was e.g. "gpt-..."
    messages=[{"role": "user", "content": "Reply with one short sentence."}],
)
print(r.choices[0].message.content)
```

**Node**

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://inference.broadnet.ai/v1",
  apiKey: process.env.BROADNET_API_KEY,
});
const r = await client.chat.completions.create({
  model: "broadnet/gemma-4-26b-a4b",
  messages: [{ role: "user", content: "Reply with one short sentence." }],
});
console.log(r.choices[0].message.content);
```

**cURL**

```bash
curl https://inference.broadnet.ai/v1/chat/completions \
  -H "Authorization: Bearer $BROADNET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"broadnet/gemma-4-26b-a4b","messages":[{"role":"user","content":"Reply with one short sentence."}]}'
```

Note the path: the base URL already ends in `/v1`, so the SDK adds `/chat/completions`. Doubling `/v1` is the most common first error.

### Can I switch without touching code?

Yes, for Chat Completions code. The Python SDK reads `OPENAI_BASE_URL` when no `base_url` is passed, and the Node client defaults `baseURL` to `process.env['OPENAI_BASE_URL']`. Both read `OPENAI_API_KEY`. Setting two environment variables moves every call.

That is also the risk. **Every call** moves, including embeddings, moderation, file uploads and anything else the new provider may not have. Use the environment switch only when your app makes chat calls and nothing else.

## What if my code uses the Responses API?

OpenAI’s SDK READMEs now call the Responses API the primary API; the Python README calls Chat Completions “the previous standard (supported indefinitely)”. Many compatible providers implement only Chat Completions. BroadNet documents `POST /v1/chat/completions` and `POST /v1/audio/transcriptions`, so Responses calls need rewriting. The mapping:

| Responses API | Chat Completions |
| --- | --- |
| `client.responses.create(...)` | `client.chat.completions.create(...)` |
| `instructions="..."` | First message `{"role": "system", "content": "..."}` |
| `input="..."` or a list of items | `messages=[...]` |
| `response.output_text` | `response.choices[0].message.content` |
| `max_output_tokens` | `max_tokens` (BroadNet’s docs) or `max_completion_tokens` |
| `text={"format": {...}}` | `response_format={...}` |
| `previous_response_id`, conversations | Resend the full message history yourself |
| Tool `{"type": "function", "name": ...}` | Tool `{"type": "function", "function": {"name": ...}}` |
| Output item `function_call`, reply with `function_call_output` and `call_id` | `message.tool_calls`, reply with role `tool` and `tool_call_id` |
| Built-in tools (web search, file search) | Not available; run your own tool |

The before and after for a simple call:

```python
# Before: Responses API
r = client.responses.create(model="gpt-...", instructions="Be brief.", input="What is RAG?")
text = r.output_text

# After: Chat Completions
r = client.chat.completions.create(
    model="broadnet/gemma-4-26b-a4b",
    messages=[{"role": "system", "content": "Be brief."},
              {"role": "user", "content": "What is RAG?"}],
)
text = r.choices[0].message.content
```

The biggest structural change is state. Responses can keep conversation state on OpenAI’s side with `previous_response_id`. Chat Completions has no server-side state, so your code stores the message list and sends all of it each turn. Budget for that: the whole history is input tokens on every call.

## How do tool calls change?

In Chat Completions, a tool definition nests the function under a `function` key, and the reply goes back as a `tool` message:

```python
import json

tools = [{
    "type": "function",
    "function": {
        "name": "get_order",
        "description": "Look up an order by ID.",
        "parameters": {"type": "object",
                       "properties": {"order_id": {"type": "string"}},
                       "required": ["order_id"]},
    },
}]
messages = [{"role": "user", "content": "Where is order A-1042?"}]
r = client.chat.completions.create(model="broadnet/glm-5.3", messages=messages, tools=tools)
msg = r.choices[0].message
if msg.tool_calls:
    messages.append(msg)                                   # keep the assistant turn
    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)         # a JSON string: parse and validate it
        messages.append({"role": "tool", "tool_call_id": call.id,
                         "content": json.dumps(lookup_order(args["order_id"]))})
    r = client.chat.completions.create(model="broadnet/glm-5.3", messages=messages, tools=tools)
```

The model matters more than the shape. BroadNet’s docs list tools for [GLM 5.3](https://broadrouter.com/models/glm-5.3/) and [Qwen3.8 27B](https://broadrouter.com/models/qwen3.8-27b/) only. Don’t point a tool loop at a model whose docs don’t list tools, and validate arguments before you run anything. [Tool calling with open models](https://broadrouter.com/blog/tool-calling-open-models/) has a full validating loop.

## What about structured output?

`client.chat.completions.parse(..., response_format=MyPydanticModel)` and Node’s `zodResponseFormat` both send `response_format` with a JSON schema. They work only if the provider accepts that shape on that model. BroadNet lists structured output for GLM 5.3 and Qwen3.8 27B but doesn’t name the request parameter. Run the four-step test in [JSON mode vs json\_schema](https://broadrouter.com/blog/structured-output-json-schema/) once per model, and keep validating the parsed result whatever the provider promises.

## What happens to embeddings?

If you change the base URL on a shared client and the provider has no embeddings endpoint, every `client.embeddings.create` call fails. BroadNet does not sell [embeddings](https://broadrouter.com/glossary/embeddings/). Keep two clients:

```python
chat = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])
embed = OpenAI(api_key=os.environ["OPENAI_API_KEY"])   # or any embeddings provider

vec = embed.embeddings.create(model="text-embedding-3-small", input=query).data[0].embedding
answer = chat.chat.completions.create(model="broadnet/glm-5.3", messages=build_prompt(query, search(vec)))
```

Never mix vectors from two embedding models in one index. If you change embedding provider too, re-embed the whole corpus. The [embedding cost calculator](https://broadrouter.com/tools/embedding-cost-calculator/) prices that.

The same applies to anything else that isn’t chat: image generation, text to speech, moderation, the Batch API and file storage. BroadNet offers none of them. Speech to text does move: `client.audio.transcriptions.create` works against `/v1/audio/transcriptions` with WAV or MP3 and a `language` field ([Arabic speech-to-text guide](https://broadrouter.com/guides/arabic-speech-to-text-api/)).

## How does streaming code change?

The Responses API streams typed events, such as `response.output_text.delta` and `response.completed`. Chat Completions streams chunks, and the text sits in `choices[0].delta.content`. Usage is not sent unless you ask for it, and then it arrives in a final chunk with an empty `choices` list.

```javascript
// Before (Responses): for await (const e of stream) if (e.type === "response.output_text.delta") out(e.delta);
const stream = await client.chat.completions.create({
  model: "broadnet/gemma-4-26b-a4b",
  messages: [{ role: "user", content: "Count from one to three." }],
  stream: true,
  stream_options: { include_usage: true },
});
for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content;
  if (text) process.stdout.write(text);
  if (chunk.usage) console.log("\ntokens:", chunk.usage.total_tokens);   // last chunk
}
```

Two habits keep stream code safe after the switch. Guard `choices[0]`, because the usage chunk has no choices. And treat a stream that ends before the final `data: [DONE]` line as incomplete; BroadNet’s docs say to record the `X-Request-ID` and decide whether a new request is safe. [Streaming chat completions](https://broadrouter.com/guides/streaming-chat-completions/) has the full pattern in Python.

## Do error handling and retries change?

The SDK’s error classes stay the same: `AuthenticationError` for 401, `RateLimitError` for 429, `APIStatusError` for the rest. What changes is what each status means at the new provider. On BroadNet, a 401 can mean an exhausted pay-as-you-go allowance, not only a wrong key, and 402 means insufficient balance for a purchase. Neither clears with a retry. A 429 can come from the rate, concurrency, token-rate or spend limit. Retry 429 and 5xx with backoff, and fix everything else. The [errors and retries guide](https://broadrouter.com/guides/errors-and-retries/) has the full table.

The SDK retries connection errors, 408, 409, 429 and 5xx twice by default. If you add your own retry loop, set `max_retries=0` on the client so the two don’t stack.

## Which parameters behave differently?

| Parameter | What to check |
| --- | --- |
| `max_tokens` / `max_completion_tokens` | OpenAI deprecates `max_tokens`; BroadNet’s docs use it. On reasoning models it includes thinking, so small values return empty answers ([why](https://broadrouter.com/guides/reasoning-tokens-empty-response/)) |
| `reasoning_effort` | BroadNet documents `low`; test other values before relying on them |
| `stream_options: {"include_usage": true}` | Documented by BroadNet; usage arrives in the last chunk |
| `n`, `logprobs`, `seed`, `logit_bias` | Not in BroadNet’s docs; test, or remove |
| `store`, `metadata` | OpenAI storage features; drop them |
| `image_url` content parts | Only on models listed with images: DeepSeek V4.1 Flash, Qwen3.8 27B, Gemma 4 VL 26B |

## How do you test the switch?

Run a short smoke test against the new provider before any traffic moves. This one checks plain chat, streaming usage, the reasoning budget and authentication. Add a tool-call case if you use tools.

```python
import os, openai
from openai import OpenAI

client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"], max_retries=0)
FAST, REASONING = "broadnet/gemma-4-26b-a4b", "broadnet/qwen3.8-27b"
msg = [{"role": "user", "content": "Reply with the word ok."}]

# 1. Plain chat returns text and usage
r = client.chat.completions.create(model=FAST, messages=msg)
assert r.choices[0].message.content and r.usage.prompt_tokens > 0

# 2. Streaming reports usage in the last chunk
chunks = list(client.chat.completions.create(model=FAST, messages=msg, stream=True,
                                             stream_options={"include_usage": True}))
assert chunks[-1].usage is not None

# 3. A reasoning model needs room: a tiny budget can come back empty
r = client.chat.completions.create(model=REASONING, messages=msg, max_tokens=2000)
assert r.choices[0].finish_reason == "stop" and r.choices[0].message.content

# 4. A bad key raises AuthenticationError (401), which you should not retry
bad = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key="invalid", max_retries=0)
try:
    bad.chat.completions.create(model=FAST, messages=msg)
    raise AssertionError("expected 401")
except openai.AuthenticationError:
    pass
print("smoke test passed")
```

Then compare quality, not just shape. Replay a sample of real prompts on both providers and score the answers; [evaluate an LLM before switching](https://broadrouter.com/blog/evaluate-llm-before-switching/) shows how. The [OpenAI to BroadNet migration tool](https://broadrouter.com/tools/openai-to-broadnet-migration/) helps you rewrite a snippet and lists which features it uses.

## What breaks most often?

-   **A global base URL** that also moves embeddings and file calls.
-   **Responses-only helpers** such as `output_text` left in code that now calls Chat Completions.
-   **Model IDs.** `gpt-...` names don’t exist elsewhere. On BroadNet they look like `broadnet/glm-5.3`.
-   **Empty replies** from reasoning models with a small `max_tokens`.
-   **Rate limits.** A BroadNet pay-as-you-go key allows 30 requests a minute and 4 in flight. Bulk jobs need a queue ([429 handling](https://broadrouter.com/blog/handle-llm-api-429-rate-limits/)).

## Where BroadNet Inference fits

BroadNet Inference suits apps whose OpenAI usage is chat and transcription: assistants, extraction, summarisation, translation and Arabic speech. It also suits apps that want flat-rate [DeepSeek V4.1 Flash](https://broadrouter.com/models/deepseek-v4.1-flash/) plans for agents. It is the wrong target if you depend on the Responses API’s server-side state, built-in tools, embeddings, image generation, text to speech, realtime voice or the Batch API. Move those to providers that offer them, or keep them on OpenAI and move only the chat calls. Gemma 4 VL 26B and both transcription models run on the $5 starting credit; GLM 5.3, Qwen3.8 27B and DeepSeek V4.1 Flash need a top-up first.

Free tools for this

-   [OpenAI to BroadNet migration helper](https://broadrouter.com/tools/openai-to-broadnet-migration/) Paste OpenAI SDK code; get base URL, key and model edits, a diff and the checks that remain.
-   [max\_tokens calculator for reasoning models](https://broadrouter.com/tools/reasoning-token-budget/) Answer length + reasoning budget + margin = a max\_tokens that does not cut the answer off.
-   [JSON schema generator for LLM structured output](https://broadrouter.com/tools/json-schema-generator/) Sample JSON to strict schema, Pydantic, Zod and a ready request.
-   [LLM JSON output validator](https://broadrouter.com/tools/llm-json-output-validator/) Parse errors by line and column, safe repairs, cut-off detection and JSON Schema checks for model output.

[All free tools](https://broadrouter.com/tools/)

## Terms in this article

[Chat Completions API](https://broadrouter.com/glossary/chat-completions-api/)

The Chat Completions API is OpenAI's HTTP endpoint, POST /v1/chat/completions, that takes a model name and a list of role-tagged messages and returns the model's next message. Its request and response shape became the common format that OpenAI-compatible providers and SDKs reuse.

[Embeddings](https://broadrouter.com/glossary/embeddings/)

Embeddings are lists of floating-point numbers (vectors) that a model produces to represent a piece of text, an image or other input. Inputs with similar meaning get vectors that are close together, so distance between vectors can rank search results, cluster documents or find related items.

[Tool calling](https://broadrouter.com/glossary/tool-calling/)

Tool calling, also called function calling, is an LLM API feature in which the model returns a structured request to run a function the developer has defined, with JSON arguments. The application runs the function and sends the result back; the model never runs code itself.

[Base URL](https://broadrouter.com/glossary/base-url/)

A base URL is the fixed root address of an API, such as https://api.example.com/v1, that a client puts in front of every endpoint path, such as /chat/completions. In OpenAI-compatible SDKs, changing the base URL, together with the API key and model ID, points the same code at a different provider.

[Pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/)

Pay-as-you-go is a pricing model in which you pay only for what you use, metered per unit such as tokens, requests or audio minutes, with no subscription or minimum commitment. Most LLM APIs bill this way, from a prepaid balance or a periodic invoice.

[Rate limit](https://broadrouter.com/glossary/rate-limit/)

A rate limit is a cap an API places on how much a client can use it in a time window, such as requests per minute or tokens per minute. Requests over the limit are refused, usually with HTTP 429 Too Many Requests, until usage falls back under it.

[Streaming](https://broadrouter.com/glossary/streaming/)

Streaming is a way of returning a language model response in small chunks while it is still being generated, instead of as one reply at the end. OpenAI-compatible APIs stream over server-sent events when the request sets stream to true.

[Structured output](https://broadrouter.com/glossary/structured-output/)

Structured output is an LLM API feature that constrains a model's reply to a format the developer specifies, usually a JSON Schema, so code can parse it reliably. It is stricter than JSON mode, which guarantees valid JSON but not a particular shape.

## Questions

### Do I need a different SDK to use an OpenAI-compatible API?

No. Keep the official openai package and pass a different base\_url (Python) or baseURL (Node), plus the new provider's key. The request and response objects stay the same for the endpoints the provider supports.

### Does an OpenAI-compatible API support the Responses API?

Some do; many support only Chat Completions. Check the provider's endpoint list. BroadNet Inference documents /v1/chat/completions and /v1/audio/transcriptions, so Responses code must be rewritten to Chat Completions.

### What happens to my embeddings when I switch?

They break if the new provider has no embeddings endpoint and you changed the base URL globally. BroadNet does not sell embeddings, so keep an embeddings client pointed at your embeddings provider. Never mix vectors from two embedding models in one index.

### Should I send max\_tokens or max\_completion\_tokens?

OpenAI marks max\_tokens as deprecated in favour of max\_completion\_tokens. Many compatible providers document max\_tokens instead, and BroadNet's docs use max\_tokens. Send what the provider documents and confirm the limit takes effect in your smoke test.

### Will OpenAI's structured output helpers still work?

Only if the provider accepts response\_format with json\_schema on the model you use. BroadNet lists structured output for GLM 5.3 and Qwen3.8 27B but does not name the request parameter, so test it, and keep validating the parsed result.

## Keep reading

-   [Switching to an OpenAI-compatible API: a migration checklist](https://broadrouter.com/blog/openai-compatible-api-migration-checklist/)
-   [OpenAI-compatible API base URLs: 14 providers, checked](https://broadrouter.com/blog/openai-compatible-api-base-urls/)
-   [Tool calling with open models: schemas, parallel calls, retries](https://broadrouter.com/blog/tool-calling-open-models/)
-   [JSON mode vs json\_schema: structured output that holds up](https://broadrouter.com/blog/structured-output-json-schema/)
-   [How to evaluate an LLM before switching providers](https://broadrouter.com/blog/evaluate-llm-before-switching/)
-   [A private endpoint starts with a clear workload](https://broadrouter.com/blog/private-endpoint-planning/)
-   [Before fine-tuning: define the problem to solve](https://broadrouter.com/blog/fine-tuning-readiness/)
-   [Guide: Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/)
-   [Guide: Use the OpenAI Node.js SDK with BroadNet Inference](https://broadrouter.com/guides/openai-node-sdk/)
-   [Guide: First request to BroadNet Inference with curl](https://broadrouter.com/guides/curl-quickstart/)
-   [Guide: Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-migrate-from-openai-api-code-walkthrough&utm_content=page-cta) ·

**Sources**

-   [BroadNet Inference API reference: endpoints, model IDs, capabilities, streaming usage, errors (checked 2026-11-15)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-migrate-from-openai-api-code-walkthrough&utm_content=sources)
-   [openai-python README: Responses is the primary API, Chat Completions supported indefinitely, retries (checked 2026-11-15)](https://github.com/openai/openai-python)
-   [openai-node client source: baseURL defaults to OPENAI\_BASE\_URL (checked 2026-11-15)](https://github.com/openai/openai-node/blob/master/src/client.ts)
-   [OpenAI, Migrate to the Responses API: differences in structured output, tools and storage (checked 2026-11-15)](https://developers.openai.com/api/docs/guides/migrate-to-responses)
-   [OpenAI function calling guide: tool shapes in both APIs (checked 2026-11-15)](https://developers.openai.com/api/docs/guides/function-calling)
-   [OpenAI Chat Completions reference: max\_tokens deprecated in favour of max\_completion\_tokens (checked 2026-11-15)](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create)
