BroadRouter
Get started

Tool calling

Updated

Tool calling, also called function calling, is an LLM API feature in which the model returns a structured request to run a function the developer has defined, with JSON arguments. The application runs the function and sends the result back; the model never runs code itself.

Also called: function calling, tool use, tools, tool_calls

Tool calling is how a language model acts on the world: it asks your code to look something up, run a query or edit a file, and then uses the result.

How it works

OpenAI’s guide describes a loop between your app and the model:

  1. Send tools. The request lists functions with a name, a description and a JSON Schema for the arguments.
  2. Get a tool call. If the model needs a tool, it returns the function name and JSON arguments instead of a final answer.
  3. Run it. Your code executes the function. OpenAI’s guide puts this step on “the application side”; the model never runs code itself.
  4. Return the result with the matching call ID.
  5. Get the answer, or another tool call. Repeat as needed.

In the Chat Completions format, a tool definition looks like this:

{
  "type": "function",
  "function": {
    "name": "get_order_status",
    "description": "Look up an order by ID",
    "parameters": {
      "type": "object",
      "properties": { "order_id": { "type": "string" } },
      "required": ["order_id"]
    }
  }
}

The reply has finish_reason: "tool_calls" and a tool_calls list. Each entry has an id, the function name and arguments as a JSON string. You append a message with role: "tool", the same tool_call_id and the result, then call the API again.

Why it matters

  • Agents. Coding agents, support bots and data assistants are mostly loops of tool calls.
  • Cost. Every round trip is a new request that resends the conversation, tool definitions and tool results as input tokens. Long tool loops make cached input and a stable tool order important.
  • Safety. Arguments come from the model. Validate them before running anything that changes data or spends money.

For a reply that must match a schema but needs no function, structured output is the simpler tool.

On BroadNet Inference

BroadNet’s docs list tool calling for GLM 5.3 and Qwen3.8 27B, both with structured output and reasoning. The docs do not state tool support for DeepSeek V4.1 Flash or BroadNet Gemma 4 VL 26B. Test your own tool loop on the model you plan to ship.

Both listed models reason before answering, and reasoning counts toward max_tokens, so leave room or a tool call can be cut off. See reasoning tokens. For agent tools that bring their own tool format, see the Cline guide and the Vercel AI SDK guide.

Common mistakes

  • Parsing arguments as an object. It arrives as a JSON string; decode it and handle errors.
  • Forgetting the tool_call_id. The model cannot match results to calls without it.

Questions

Does the model run my function?

No. The model only returns the function name and arguments. Your code decides whether to run it, runs it, and sends the result back in a tool message.

Can the model call several tools at once?

Many APIs allow parallel tool calls, returning several entries in tool_calls in one response. Your code runs each and returns one result per tool_call_id.

Which BroadNet models support tool calling?

BroadNet's docs list tools for GLM 5.3 and Qwen3.8 27B. They do not state tool support for DeepSeek V4.1 Flash or BroadNet Gemma 4 VL 26B, so test before you rely on it there.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity