BroadRouter

Use LangChain with BroadNet Inference

Updated

Install langchain-openai and create ChatOpenAI with base_url set to https://inference.broadnet.ai/v1, api_key set to your BroadNet key, and a BroadNet model ID such as broadnet/glm-5.3. Invoke, stream and tool-calling chains then run over chat completions. Avoid Responses-only features, and leave max_tokens room for reasoning.

LangChain’s ChatOpenAI accepts any endpoint through base_url. This works with BroadNet Inference because it uses the standard chat-completions shape. If you call the API directly instead, see the Python SDK guide.

Install

Current langchain-openai releases (1.x) need Python 3.11 or newer.

pip install -U langchain-openai
export BROADNET_API_KEY='paste-the-key-once'

Connect

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="broadnet/glm-5.3",
    base_url="https://inference.broadnet.ai/v1",
    api_key=os.environ["BROADNET_API_KEY"],
    max_tokens=2000,  # reasoning counts toward this limit
)

print(llm.invoke("Summarise the CAP theorem in two sentences.").content)

openai_api_base is an alias for base_url. Without either, LangChain reads OPENAI_API_BASE, and then the underlying openai client reads OPENAI_BASE_URL.

Things to know

  • Reasoning text isn’t kept. LangChain’s docs say ChatOpenAI follows the official OpenAI specification and does not keep non-standard fields such as reasoning_content. You get the final answer, which is usually what a chain needs.
  • Leave room for reasoning. GLM 5.3, DeepSeek V4.1 Flash and Qwen3.8 27B reason first. A small max_tokens can return an empty answer. Why answers come back empty.
  • Streaming usage. When you configure the endpoint with OPENAI_BASE_URL, LangChain turns off stream_usage by default. Pass stream_usage=True if you want token counts while streaming. BroadNet’s docs say usage arrives in the last chunk when it’s requested.
  • Stay on chat completions. BroadNet serves /v1/chat/completions, not the Responses API. LangChain’s docs say ChatOpenAI switches to the Responses API when you use OpenAI built-in tools (web search, file search, code interpreter and similar), conversation state, or the reasoning parameter. Leave those out, and don’t set use_responses_api=True.
  • Token limit name. LangChain’s docs say max_tokens is converted to max_completion_tokens internally. BroadNet’s docs describe max_tokens. If a limit seems ignored, check the request your app sends.

Which model for which chain

Chain Model
Tool calling and structured output broadnet/glm-5.3
Long documents and agents, unlimited plan broadnet/deepseek-v4.1-flash
Arabic text or images broadnet/qwen3.8-27b

See prices for every model.

Troubleshooting

  • 401. The key is wrong, expired, or out of pay-as-you-go allowance. If you passed no api_key, LangChain used OPENAI_API_KEY, which may hold an OpenAI key.
  • 404 or “not found”. Check that base_url ends in /v1 and that nothing routes the call to the Responses API (see above). Then check the model ID matches BroadNet’s exactly, with the broadnet/ prefix. Paid models need a top-up; the $5 credit covers broadnet/gemma-4-26b-a4b and the two Transcribe models.
  • Empty content. Reasoning used the budget. Raise max_tokens. Why answers come back empty.
  • Tool calls never happen. Use broadnet/glm-5.3 or broadnet/qwen3.8-27b. BroadNet’s docs list tools only for those two.
  • 429 in batch or parallel chains. A pay-as-you-go key allows 30 requests a minute and 4 at once. Set max_concurrency in the runnable config for batch(). See error codes and retries.

Questions

Should I use base_url or openai_api_base in ChatOpenAI?

Either. openai_api_base is an alias for base_url. If neither is passed, LangChain checks OPENAI_API_BASE, and then the openai client reads OPENAI_BASE_URL.

Can I set the BroadNet endpoint with environment variables only?

Yes. Set OPENAI_BASE_URL to https://inference.broadnet.ai/v1 and OPENAI_API_KEY to your BroadNet key, then create ChatOpenAI with only the model ID. Pass stream_usage=True if you want token counts while streaming.

Why does LangChain drop reasoning_content from BroadNet responses?

LangChain's docs say ChatOpenAI follows the official OpenAI specification and does not extract non-standard fields such as reasoning_content. You still get the final answer in content.

Does ChatOpenAI with BroadNet support tool calling?

Yes, through bind_tools, on models BroadNet lists with tools. Use broadnet/glm-5.3 or broadnet/qwen3.8-27b for tool calling and with_structured_output.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity