Vocabulary size tells you how many different pieces of text a model can represent with a single token.
How it works
A BPE tokenizer’s vocabulary is its base symbols plus every merge it learned. Hugging Face’s documentation gives the original GPT as an example: 478 base tokens + 40,000 merges = 40,478. The builder picks the target size before training.
We read the vocabularies from each model’s pinned tokenizer.json on Hugging Face on 2026-11-14:
| Tokenizer | Vocabulary entries | Total IDs incl. special tokens |
|---|---|---|
| DeepSeek V4.1 Flash | 128,000 | 129,280 |
| GLM 5.3 | 154,820 | 154,856 |
| Qwen3.8 27B | 248,044 | 248,077 |
| Gemma 4 26B A4B (Google base) | 262,144 | 262,144 |
| OpenAI o200k_base (reference) | 199,998 | 200,019 |
The Gemma 4 model card states a 262K vocabulary, which matches. The Qwen3.8 card lists an embedding size of 248,320 “padded”, larger than the 248,077 IDs actually used.
Why it matters
For an API user, vocabulary size is an indirect hint about cost per word. What counts is which text the merges were trained on. Compare tokens per 100 words for the same 39,037 words of Modern Standard Arabic in our tokenizer dataset:
- DeepSeek V4.1 Flash, 128,000 entries: 86,690 / 39,037 x 100 = 222.1
- GLM 5.3, 154,820 entries: 95,230 / 39,037 x 100 = 243.9
- Qwen3.8 27B, 248,044 entries: 71,424 / 39,037 x 100 = 183.0
- Gemma 4, 262,144 entries: 79,120 / 39,037 x 100 = 202.7
GLM 5.3 has more entries than DeepSeek but uses more tokens on this text. Gemma has the most entries but is not the most compact. For English prose all five land between 127 and 132 tokens per 100 words, so vocabulary size barely matters there.
The rule: measure your own text with the model’s tokenizer rather than inferring cost from the vocabulary number.
On BroadNet Inference
BroadNet does not publish vocabulary sizes; the figures above come from the public model repos. Gemma’s row uses Google’s base tokenizer, which we assume BroadNet’s Gemma 4 VL 26B build keeps. To see the effect on your own prompts, use the token counter, and for model choice in Arabic read choosing an LLM for Arabic alongside the tokens per word study.