---
title: "What is a chat template? How messages become LLM tokens"
description: "A chat template turns a list of role and content messages into the exact token sequence a chat model was trained on. How templates work and why the wrong one hurts."
url: https://broadrouter.com/glossary/chat-template/
lang: en
updated: 2026-11-15
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Chat template

Updated 2026-11-15

A chat template is the formatting rule, usually written in Jinja and shipped with a model's tokenizer, that converts a list of chat messages with roles into the single token sequence the model expects, including its special control tokens. Using the wrong template can sharply degrade a model's answers.

Also called: chat\_template, Jinja chat template, tokenizer chat template

A chat model never sees a list of messages. It sees one long run of tokens. The chat template decides what that run looks like.

## How it works

Hugging Face’s guide states the key point: all causal language models “continue a sequence of tokens”, and the role and content dictionaries you pass “get converted to a token sequence, often with control tokens”. Different models use different formats, “even if they were fine-tuned from the same base model”.

The same short chat, rendered by two templates in the guide:

```text
Mistral-7B-Instruct:
<s>[INST] Hello, how are you? [/INST]I'm doing great. How can I help you today?</s>

Zephyr-7B:
<|user|>
Hello, how are you?</s>
<|assistant|>
I'm doing great. How can I help you today?</s>
```

In Transformers you render a chat with `tokenizer.apply_chat_template(messages, add_generation_prompt=True)`. The `add_generation_prompt` flag adds the tokens that open an assistant turn; without them, the guide warns, the model may continue the user’s message instead of replying.

Templates can carry model features too. Google’s Gemma 4 card says thinking is turned on by putting a `<|think|>` token at the start of the system prompt, and that “many libraries like Transformers and llama.cpp handle the complexities of the chat template for you.”

## Why it matters

-   **Quality.** The guide says that “with the wrong control tokens, these models would have drastically worse performance.”
-   **Self-hosting.** Serving engines need a template to accept chat requests. vLLM’s docs say that without one “all chat requests will error”, and accept `--chat-template` to supply it. See [vLLM](https://broadrouter.com/glossary/vllm/).
-   **Token counts.** Control tokens count as input [tokens](https://broadrouter.com/glossary/token/). Our post on [counting tokens before sending](https://broadrouter.com/blog/count-tokens-before-sending/) covers why local counts and billed counts differ.
-   **Double special tokens.** The guide warns that adding `<bos>` or `<eos>` again after templating “is often incorrect or duplicated, hurting model performance”.
-   **The [system prompt](https://broadrouter.com/glossary/system-prompt/) is part of it.** Where the system message lands, and whether a model has a system role at all, is decided by the template.

## On BroadNet Inference

BroadNet’s API is an [OpenAI-compatible](https://broadrouter.com/glossary/openai-compatible-api/) chat completions endpoint, so you send `messages` and BroadNet applies the template; you do not send raw template text. BroadNet reports that it serves [Gemma 4 VL 26B](https://broadrouter.com/models/gemma-4-vl-26b/) with a “BroadNet conversational chat template”, a template tuned for its conversational workflow. If you compare BroadNet’s Gemma with the same weights run locally, answers may differ, so test with your own prompts. BroadNet does not publish template details for its other models. The [GPU fleet and serving post](https://broadrouter.com/blog/broadnet-gpu-fleet-and-serving/) lists its other Gemma serving choices.

## Questions

### Do I need to apply a chat template when I call an API?

No. With a chat completions API you send a messages list and the provider applies the template on its side. You only handle templates yourself when you run a model locally or call a raw text-completion endpoint.

### Why do my token counts differ from the provider's?

The template adds control tokens around every message. Counting only your message text misses them, so the billed prompt is usually a little larger.

### What does add\_generation\_prompt do?

It appends the tokens that start an assistant turn, so the model replies instead of continuing your message. Hugging Face's docs show this as a trailing <|im\_start|>assistant in ChatML-style templates.

## See also

-   [Tokenizer](https://broadrouter.com/glossary/tokenizer/) (term)
-   [System prompt](https://broadrouter.com/glossary/system-prompt/) (term)
-   [Token](https://broadrouter.com/glossary/token/) (term)
-   [vLLM](https://broadrouter.com/glossary/vllm/) (term)
-   [OpenAI-compatible API](https://broadrouter.com/glossary/openai-compatible-api/) (term)
-   [GGUF](https://broadrouter.com/glossary/gguf/) (term)
-   [How to count tokens before you send an LLM request](https://broadrouter.com/blog/count-tokens-before-sending/) (article)
-   [BroadNet's GPU hardware and serving approach, as BroadNet describes it](https://broadrouter.com/blog/broadnet-gpu-fleet-and-serving/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-chat-template&utm_content=page-cta) ·

**Sources**

-   [Hugging Face Transformers docs, chat templates (checked 2026-11-15)](https://huggingface.co/docs/transformers/en/chat_templating)
-   [vLLM documentation, online serving: chat template (checked 2026-11-15)](https://docs.vllm.ai/en/latest/serving/online_serving/)
-   [Google, Gemma 4 26B A4B instruction-tuned model card (checked 2026-11-15)](https://huggingface.co/google/gemma-4-26B-A4B-it)
-   [BroadNet, Gemma 4 VL 26B model page: serving configuration (checked 2026-11-15)](https://www.broadnet.ai/inference/models/gemma-4-vl-26b)
