A chat model never sees a list of messages. It sees one long run of tokens. The chat template decides what that run looks like.
How it works
Hugging Face’s guide states the key point: all causal language models “continue a sequence of tokens”, and the role and content dictionaries you pass “get converted to a token sequence, often with control tokens”. Different models use different formats, “even if they were fine-tuned from the same base model”.
The same short chat, rendered by two templates in the guide:
Mistral-7B-Instruct:
<s>[INST] Hello, how are you? [/INST]I'm doing great. How can I help you today?</s>
Zephyr-7B:
<|user|>
Hello, how are you?</s>
<|assistant|>
I'm doing great. How can I help you today?</s>
In Transformers you render a chat with tokenizer.apply_chat_template(messages, add_generation_prompt=True). The add_generation_prompt flag adds the tokens that open an assistant turn; without them, the guide warns, the model may continue the user’s message instead of replying.
Templates can carry model features too. Google’s Gemma 4 card says thinking is turned on by putting a <|think|> token at the start of the system prompt, and that “many libraries like Transformers and llama.cpp handle the complexities of the chat template for you.”
Why it matters
- Quality. The guide says that “with the wrong control tokens, these models would have drastically worse performance.”
- Self-hosting. Serving engines need a template to accept chat requests. vLLM’s docs say that without one “all chat requests will error”, and accept
--chat-templateto supply it. See vLLM. - Token counts. Control tokens count as input tokens. Our post on counting tokens before sending covers why local counts and billed counts differ.
- Double special tokens. The guide warns that adding
<bos>or<eos>again after templating “is often incorrect or duplicated, hurting model performance”. - The system prompt is part of it. Where the system message lands, and whether a model has a system role at all, is decided by the template.
On BroadNet Inference
BroadNet’s API is an OpenAI-compatible chat completions endpoint, so you send messages and BroadNet applies the template; you do not send raw template text. BroadNet reports that it serves Gemma 4 VL 26B with a “BroadNet conversational chat template”, a template tuned for its conversational workflow. If you compare BroadNet’s Gemma with the same weights run locally, answers may differ, so test with your own prompts. BroadNet does not publish template details for its other models. The GPU fleet and serving post lists its other Gemma serving choices.