OpenAI’s API format became the common language of LLM tooling. Many providers now accept it, so one client library can talk to many backends.
How it works
A compatible API copies the parts of the OpenAI API that clients depend on:
- the same paths, such as
/v1/chat/completionsand/v1/audio/transcriptions; - the same JSON shapes: a
messagesarray in,choices[0].message.contentout, and ausageblock with token counts; - an API key sent as
Authorization: Bearer <key>; - streaming as server-sent events when
streamis true.
Switching is a configuration change. With the official Python SDK:
from openai import OpenAI
import os
client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])
reply = client.chat.completions.create(
model="broadnet/gemma-4-26b-a4b",
messages=[{"role": "user", "content": "Say hello in Arabic."}],
)
print(reply.choices[0].message.content)
Only three values changed from an OpenAI example: the base URL, the key and the model ID.
Why it matters
Compatibility lowers switching cost. SDKs, agent frameworks, IDE plugins and routers that speak the OpenAI format work without new code. You can test a second provider in minutes and keep the option to move.
But compatible is not identical. Anthropic’s own compatibility layer, for example, ignores strict on tools and leaves prompt_tokens_details empty. Typical gaps are:
- parameters that are ignored or rejected;
- how reasoning tokens count toward
max_tokens; - whether usage is reported at the end of a stream;
- rate limits and error codes.
On BroadNet Inference
BroadNet Inference serves POST /v1/chat/completions, POST /v1/audio/transcriptions and GET /v1/models at https://inference.broadnet.ai/v1. Keys are sent only as Authorization: Bearer <key>. The Python and Node guides show full setups, and LangChain works the same way.
Two differences to plan for: DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason before answering, and the reasoning counts toward max_tokens. And a pay-as-you-go key allows 30 requests a minute and 4 at once. The migration checklist covers the rest.
Common mistakes
- Keeping an OpenAI model name such as
gpt-6-astra(OpenAI’s flagship on its models page, checked 2026-11-15). Use the provider’s model ID. - Assuming a parameter works because the request succeeded. Some providers ignore unknown fields silently.