BroadRouter
Get started

Vision-language model

Updated

A vision-language model (VLM) is a model that takes images and text together as input and produces text, so it can describe pictures, answer questions about them and read documents, charts or screenshots. VLMs are also called image-text-to-text or multimodal models.

Also called: VLM, vision model, multimodal LLM, image-text-to-text model, vision API

A vision-language model lets you ask questions about an image the same way you ask about text.

How it works

Hugging Face describes these models simply: they “take in an image and text prompt and output text.” They are trained on image-text pairs, so they learn to connect what they see with language.

Through an OpenAI-compatible API, the image goes inside the message as a content part. It can be a public URL or a base64 data URL:

import base64, os
from openai import OpenAI

client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])
b64 = base64.b64encode(open("receipt.jpg", "rb").read()).decode()

reply = client.chat.completions.create(
    model="broadnet/qwen3.8-27b",
    max_tokens=2000,
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "Return the vendor and total as JSON."},
        {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}},
    ]}],
)
print(reply.choices[0].message.content)

The model turns the image into input tokens, reads them with your text, and writes an answer.

Why it matters

  • Documents. Invoices, receipts, forms and IDs can go straight to a model, often with structured output to get fields back as JSON.
  • Screens and charts. Agents can read screenshots and dashboards; support tools can read photos users send.
  • Cost and limits. Each image adds input tokens and fills part of the context window. Many images in one request add up quickly.

Check the result. VLMs can misread small or blurred text and can state wrong numbers confidently. For anything that moves money, validate totals in code.

On BroadNet Inference

Three chat models accept images:

Model Input Context
DeepSeek V4.1 Flash Text and image 1,048,576 tokens
Qwen3.8 27B Text and image 262,144 tokens
BroadNet Gemma 4 VL 26B Text and image 32,768 tokens

GLM 5.3 is text only. BroadNet Gemma 4 VL 26B costs $0.10 per 1M input tokens and works on the free $5 starting credit. Qwen3.8 27B was tested by BroadNet in Arabic and also supports structured output. On pay-as-you-go, an image counts as input tokens, and larger images use more.

The Arabic invoice guide walks through a full extraction. Keep the MIME type in the data URL accurate, and leave room in max_tokens on reasoning models.

Questions

Is a vision-language model the same as OCR?

No. OCR converts printed text in an image to characters. A VLM can also read text, but it understands layout and content, so it can answer questions such as which line is the total, or what a chart shows.

Do images cost tokens?

Yes. Images are converted to input tokens. Larger images usually use more, so resize them to the smallest size that keeps the detail you need.

Which BroadNet models accept images?

DeepSeek V4.1 Flash, Qwen3.8 27B and BroadNet Gemma 4 VL 26B take text and image input. GLM 5.3 is text only.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity