DSPy is a Python framework for programming language models with signatures and modules, and for optimizing the prompts behind them. You connect it to a model with dspy.LM. DSPy’s docs show OpenAI-compatible servers as an openai/ model with api_base. For BroadNet Inference, which serves chat completions at https://inference.broadnet.ai/v1, you also need model_type="chat". BroadRouter is BroadNet’s own site; this follows the DSPy docs for 3.5.
Install
pip install dspy
export BROADNET_API_KEY='your-broadnet-key'
Connect
import os
import dspy
lm = dspy.LM(
"openai/broadnet/glm-5.3",
model_type="chat",
api_base="https://inference.broadnet.ai/v1",
api_key=os.environ["BROADNET_API_KEY"],
max_tokens=4000,
)
dspy.configure(lm=lm)
qa = dspy.Predict("question -> answer")
print(qa(question="What is the capital of Saudi Arabia?").answer)
openai/tells DSPy to use its OpenAI client. The rest of the string,broadnet/glm-5.3, is the model ID sent to BroadNet.model_type="chat"keeps DSPy on/v1/chat/completions. DSPy’s LM reference says the default,'auto', makes OpenAI models “use Responses”.max_tokensleaves room for reasoning. DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B think before they answer, and the thinking counts toward the limit.
What happens without model_type
The example on DSPy’s installation page for an OpenAI-compatible server leaves model_type out. In our test with DSPy 3.5.0, that sent the request to /v1/responses with max_output_tokens, and the call failed:
dspy.utils.exceptions.LMInvalidRequestError: [openai/broadnet/glm-5.3] /v1/responses not served
That error came from our mock server. BroadNet’s live endpoint also answered /v1/responses with a 404 when we checked on 2026-11-16. Adding model_type="chat" fixed it.
Modules and models
cot = dspy.ChainOfThought("question -> answer")
print(cot(question="A train leaves at 9:40 and arrives at 13:05. How long is the trip?").answer)
| DSPy use | Model |
|---|---|
Predict, ChainOfThought, optimizer runs |
broadnet/deepseek-v4.1-flash (low per-token price; flat-price plans) |
ReAct and other tool-calling modules |
broadnet/glm-5.3 or broadnet/qwen3.8-27b |
| Trying DSPy on the starting credit | broadnet/gemma-4-26b-a4b (32,768-token context) |
BroadNet’s docs list tools and structured output for GLM 5.3 and Qwen3.8 27B only. You can use one model for the program and another for an optimizer, each with its own dspy.LM. Prices for every model.
How we tested this
On 2026-11-15 we installed dspy 3.5.0 in a clean virtual environment and ran dspy.Predict against a local mock server that speaks BroadNet’s chat-completions shape and answers /v1/responses with a 404. Without model_type, DSPy called /v1/responses and failed. With model_type="chat", it called /v1/chat/completions with the model ID broadnet/glm-5.3 and a Bearer key, and parsed the answer field. We saw that DSPy sent the output limit as max_completion_tokens. BroadNet’s API reference describes max_tokens and doesn’t mention max_completion_tokens, so we can’t confirm that BroadNet applies a limit sent that way. We tested against a mock server, not with a BroadNet key, so we did not check real model output.
Troubleshooting
- 404 on /v1/responses. Add
model_type="chat". - 404 on /v1/chat/completions.
api_basemust end in/v1, and the model string must beopenai/plus the full BroadNet ID. Paid models need a top-up; the $5 credit covers onlybroadnet/gemma-4-26b-a4band the two Transcribe models. - “Expected to find output fields”. The answer was empty or cut short, often because reasoning used the output budget. Raise
max_tokens. Why answers come back empty. - 429 during optimization. A pay-as-you-go key allows 30 requests a minute and 4 in flight. Lower the optimizer’s thread count and see errors and retries.
