---
title: "What is a token in an LLM? Tokens explained simply | BroadRouter"
description: "A token is the unit of text an LLM reads and writes, and the unit APIs bill and limit. How tokenizers split text, rough word counts, and why tokens drive cost."
url: https://broadrouter.com/glossary/token/
lang: en
updated: 2026-11-14
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Token

Updated 2026-11-14

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

Also called: LLM token, tokens, tokenization

Models do not see letters or words. They see a sequence of integers, each standing for a piece of text from a fixed vocabulary. Those pieces are tokens.

## How it works

A [tokenizer](https://broadrouter.com/glossary/tokenizer/) splits text before the model reads it. Most modern models use subword methods such as [byte pair encoding (BPE)](https://broadrouter.com/glossary/bpe/). Common words stay whole; rare words break into parts. Hugging Face’s example: `annoyingly` may become `annoying` and `ly`.

Byte-level BPE starts from 256 byte values, so any input can be encoded, including emoji and every script.

For English, OpenAI’s rule of thumb is about 4 characters or 0.75 words per token, so 100 tokens is roughly 75 words. Our own measurement of English prose on five current tokenizers came to about 129 tokens per 100 words, or 1,290 tokens per 1,000 words, close to the rule of thumb. Either ratio is only an estimate:

-   other languages, including Arabic, often split into more tokens per word on tokenizers trained mostly on English;
-   code, numbers and whitespace tokenize differently from prose;
-   every model family has its own vocabulary.

To estimate before you send, paste text into the [token counter](https://broadrouter.com/tools/token-counter/). The API reports the real count in the `usage` block of each response:

```json
"usage": { "prompt_tokens": 1200, "completion_tokens": 340, "total_tokens": 1540 }
```

## Why it matters

Tokens are the unit behind almost every number on an LLM API:

-   **Price.** Per-token APIs bill per 1M [input and output tokens](https://broadrouter.com/glossary/input-and-output-tokens/), often at different rates.
-   **Limits.** The [context window](https://broadrouter.com/glossary/context-window/) and `max_tokens` are measured in tokens, and many rate limits count tokens per minute.
-   **Speed.** Generation speed is quoted in [tokens per second](https://broadrouter.com/glossary/tokens-per-second/).

The same text can be a different number of tokens on different models, especially in Arabic. The [tokenizer efficiency dataset](https://broadrouter.com/data/tokenizer-efficiency/) measures tokens per 100 words for each model in English, Arabic and code.

## On BroadNet Inference

Chat models are priced per 1M tokens. For [DeepSeek V4.1 Flash](https://broadrouter.com/models/deepseek-v4.1-flash/), new input costs $0.175, [cached input](https://broadrouter.com/glossary/cached-input-tokens/) costs $0.0055 and output costs $0.52. [BroadNet Gemma 4 VL 26B](https://broadrouter.com/models/gemma-4-vl-26b/) costs $0.10 input and $0.22 output. Every model’s rates are on the [pricing page](https://broadrouter.com/pricing/).

On a flat plan, tokens are not metered at all: a session of DeepSeek V4.1 Flash has no token meter, and the practical limit is speed, about 100 tokens per second. Speech-to-text models bill per audio minute, not per token.

## Common mistakes

-   Estimating cost from word counts for non-English text. Measure with `usage` on real samples.
-   Forgetting that reasoning models also spend output tokens on thinking. See [reasoning tokens](https://broadrouter.com/glossary/reasoning-tokens/).

## Questions

### How many words is 1,000 tokens?

For English prose, about 775 words by BroadRouter's own measurement across five tokenizers (1,000 English words came to about 1,290 tokens). OpenAI's rule of thumb, a token is about three quarters of a word, gives 750. Other languages, code and other models' tokenizers give different ratios, so count with the model's own tokenizer when it matters.

### Do images and audio count as tokens?

Images sent to a vision model are converted to input tokens, and larger images usually use more. Speech-to-text APIs are often priced per audio minute instead of per token.

### Why does the same text use a different number of tokens on two models?

Each model family has its own tokenizer and vocabulary, so the same sentence can split into a different number of pieces.

## See also

-   [Input and output tokens](https://broadrouter.com/glossary/input-and-output-tokens/) (term)
-   [Cached input tokens](https://broadrouter.com/glossary/cached-input-tokens/) (term)
-   [Context window](https://broadrouter.com/glossary/context-window/) (term)
-   [Tokens per second](https://broadrouter.com/glossary/tokens-per-second/) (term)
-   [Tokenizer](https://broadrouter.com/glossary/tokenizer/) (term)
-   [Byte pair encoding (BPE)](https://broadrouter.com/glossary/bpe/) (term)
-   [Vocabulary size](https://broadrouter.com/glossary/vocabulary-size/) (term)
-   [Logprobs](https://broadrouter.com/glossary/logprobs/) (term)
-   [Stream chat completions and handle dropped connections](https://broadrouter.com/guides/streaming-chat-completions/) (guide)
-   [Cached input tokens: how to cut an LLM API bill](https://broadrouter.com/blog/cached-input-tokens-explained/) (article)
-   [Flat-rate vs pay-per-token LLM API: the break-even math](https://broadrouter.com/blog/flat-rate-vs-pay-per-token-llm-api/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-token&utm_content=page-cta) ·

**Sources**

-   [Hugging Face Transformers, tokenization algorithms (BPE, Unigram, WordPiece)](https://huggingface.co/docs/transformers/tokenizer_summary)
-   [OpenAI, key concepts (tokens rule of thumb)](https://developers.openai.com/api/docs/concepts)
-   [BroadNet Inference machine-readable prices (pricing.json)](https://inference.broadnet.ai/pricing.json?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-token&utm_content=sources)
