BroadRouter

Use the Vercel AI SDK with BroadNet Inference

Updated

Install ai and @ai-sdk/openai-compatible, then call createOpenAICompatible with name broadnet, baseURL https://inference.broadnet.ai/v1 and your BroadNet key as apiKey. Pass a BroadNet model ID such as broadnet/glm-5.3 to the provider, and use generateText or streamText as usual. Keep the key on the server, and give reasoning models a generous maxOutputTokens.

The AI SDK from Vercel is a TypeScript toolkit for LLM apps. Its @ai-sdk/openai-compatible package creates a provider for any API that follows the OpenAI chat-completions format. BroadNet Inference does, so you only change the base URL, key and model IDs. For the plain OpenAI client instead, see the Node SDK guide.

Setup

  1. Install the packages. At the time of writing, npm serves ai 7.x and @ai-sdk/openai-compatible 3.x.

    npm install ai @ai-sdk/openai-compatible
    export BROADNET_API_KEY='your-broadnet-key'
  2. Create the provider.

    // broadnet.ts
    import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
    
    export const broadnet = createOpenAICompatible({
      name: 'broadnet',
      baseURL: 'https://inference.broadnet.ai/v1',
      apiKey: process.env.BROADNET_API_KEY,
      includeUsage: true, // usage data in streaming responses
    });
  3. Generate text.

    import { generateText } from 'ai';
    import { broadnet } from './broadnet';
    
    const { text, usage, finishReason } = await generateText({
      model: broadnet('broadnet/glm-5.3'),
      prompt: 'Explain idempotency keys in two sentences.',
      maxOutputTokens: 4000, // reasoning counts toward this
    });
    
    console.log(text, finishReason, usage);
  4. Stream it. Swap generateText for streamText and read textStream. The streaming guide explains the wire format underneath.

The name you pass becomes the key for provider options and provider metadata, so keep it short and stable. provider('id') and provider.chatModel('id') both create a chat model; there is no separate setup per model. Keep the key on the server. Don’t import this file into code that runs in the browser, or the key ships to every visitor.

The provider docs list text generation, streaming, tool calling, structured outputs, reasoning content and image input as supported capabilities, with the note that some depend on the provider. On BroadNet, image input works with DeepSeek V4.1 Flash, Qwen3.8 27B and Gemma 4 VL 26B. GLM 5.3 takes text only.

Options worth setting

  • reasoningEffort. Pass it through providerOptions, keyed by the provider name. The AI SDK docs say the exact values depend on the provider; BroadNet’s docs show low.
const result = await generateText({
  model: broadnet('broadnet/deepseek-v4.1-flash'),
  prompt: 'Classify: "refund my order" -> intent?',
  providerOptions: { broadnet: { reasoningEffort: 'low' } },
});
  • supportsStructuredOutputs. The AI SDK only sends JSON schemas when this is true on the provider. BroadNet lists structured output for GLM 5.3 and Qwen3.8 27B. If you turn it on, use those models for schema output.
  • maxRetries. generateText retries twice by default. Set it to fit your rate limit.
  • Reasoning output. The AI SDK reads reasoning content when the provider returns it. The top-level reasoningText result field is now marked deprecated in favour of finalStep.reasoningText. Don’t build logic that depends on reasoning being present.

Which model to pick

Need Model Context
Tool calling, structured output broadnet/glm-5.3 1,048,576
Images plus tools, Arabic broadnet/qwen3.8-27b 262,144
Long context, flat-rate plan broadnet/deepseek-v4.1-flash 1,048,576
Cheap text and image calls broadnet/gemma-4-26b-a4b 32,768

Per-token prices for each model are on the pricing page.

Troubleshooting

  • 401. BROADNET_API_KEY is missing in the runtime (check your hosting provider’s env settings), wrong, expired, or out of pay-as-you-go allowance.
  • providerOptions ignored. The key must match the provider name in camelCase. With name: 'broadnet', use providerOptions.broadnet.
  • Model errors. The ID must match BroadNet’s exactly, with the broadnet/ prefix. GLM 5.3, Qwen3.8 27B and DeepSeek V4.1 Flash need a top-up; the $5 starting credit covers Gemma 4 VL 26B and the two Transcribe models.
  • Empty text, finishReason “length”. Reasoning used the budget. Raise maxOutputTokens or lower reasoningEffort. Why answers come back empty.
  • 429 under load. A pay-as-you-go key allows 30 requests a minute and 4 in flight. Serverless functions scale out fast, so cap concurrency in your app. See errors and retries.
  • Agents in parallel on a plan key. One unlimited plan session runs one request at a time. Running Promise.all over several agents needs one session per agent, or the requests queue.

Questions

Should I use @ai-sdk/openai or @ai-sdk/openai-compatible?

Use @ai-sdk/openai-compatible for third-party OpenAI-compatible APIs. It calls the chat completions endpoint, which is what BroadNet Inference serves.

Why is the generated text empty?

Reasoning counts toward max_tokens on BroadNet. If maxOutputTokens is small, the model can use it all before the answer starts. Raise it or set reasoningEffort to low.

How do I get token usage while streaming?

Set includeUsage to true when you create the provider. The AI SDK docs say it adds usage data to streaming responses. It defaults to false.

Do I need a separate setup for each BroadNet model?

No. One createOpenAICompatible provider covers every model. Call broadnet('model-id') or broadnet.chatModel('model-id') with any BroadNet chat model ID.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity