BroadRouter

Arabic token counter

Updated

In short: Modern Standard Arabic needs 1.31× to 1.77× the tokens of the same sentences in English, depending on the tokenizer (five real tokenizers, our open dataset). An Arabic prompt therefore costs more than its English translation at the same per-token price. Paste Arabic, and its translation if you have one, to see tokens, the ratio and the cost per model.

Free, no sign-up, runs in your browser: nothing you type is sent anywhere.

Samples:

ModelArabic tokensEnglish tokensArabic taxArabic as inputExtra vs English

Arabic tax = Arabic tokens ÷ English tokens for the same meaning. Costs use each model's price per 1M input tokens. ≈ marks an estimate until that tokenizer is loaded.

Create an account ($5 starting credit)

Report a wrong price or number (email to info@broadnet.ai, with this page's link and your inputs filled in).

How it works

  1. Paste Arabic text into the right-to-left box, or pick a sample. Optionally paste its English translation in the second box.
  2. The table counts tokens with each model's tokenizer. Load the real tokenizers to replace the estimates marked ≈.
  3. Read the Arabic-to-English ratio (the "Arabic tax") and the extra cost of sending the Arabic version as input.
  4. Copy the link to share the comparison, or download CSV.

The Arabic tax, measured

We counted 6,076 Arabic sentences, in Modern Standard Arabic and three dialects, and their English translations with each real tokenizer: 2,009 FLORES-200 sentences each for MSA, Egyptian and Levantine, and 49 Tatoeba sentences for Gulf Arabic, which FLORES-200 does not cover (so the Gulf column is less precise). Arabic tax = Arabic tokens ÷ English tokens for the same sentences.

TokenizerMSA taxGulf taxEgyptian taxLevantine taxMSA tokens per wordInput price per 1M
DeepSeek V4.1 Flash1.62×1.44×1.63×1.58×2.22$0.175
GLM 5.31.77×1.57×1.75×1.70×2.44$0.78
Qwen3.8 27B1.31×1.21×1.35×1.29×1.83$0.175
Gemma 4 VL 26B1.47×1.22×1.44×1.41×2.03$0.10
GPT (o200k)1.38×1.32×1.41×1.35×1.89—

Measured 2026-11-15. Parallel text: FLORES-200 dev and devtest (2,009 aligned sentences per language, CC BY-SA 4.0, tarball pinned by SHA-256) for Modern Standard, Egyptian, North Levantine, Najdi and Mesopotamian Arabic, Hindi, Urdu, Chinese, French and Spanish against English, each sentence counted on its own; Tatoeba sentence pairs (CC BY 2.0 FR) for Gulf Arabic and Arabizi. Ratio = pooled tokens ÷ pooled tokens of the English side, with a 95% interval from a seeded bootstrap over sentences (10,000 resamples). Other texts: Wikipedia lead sections (CC BY-SA 4.0), Project Gutenberg (public domain) and Wikidata JSON (CC0), stored with source URL, licence and revision. Tokens: the official tokenizer file at a pinned Hugging Face commit, encoded with @huggingface/tokenizers with no special tokens; o200k_base via gpt-tokenizer; Kimi's tiktoken rank file with a tiktoken-compatible counter. Gated Meta and Google files were read from ungated mirrors proven byte-identical to the official file. Every new count matched the Rust tokenizers library or Python tiktoken. Words: word-like segments from Intl.Segmenter. Characters: Unicode code points. Every text, source and count is in the tokenizer efficiency dataset (CSV and JSON).

On Modern Standard Arabic the tax ranges from 1.31× (Qwen3.8 27B) to 1.77× (GLM 5.3). English counts are close across tokenizers, so the difference comes from how much Arabic each vocabulary learned. Dialects land near MSA, but dialect spellings that a vocabulary never saw split into more pieces.

The formula

Arabic tax          = Arabic tokens ÷ English tokens (same meaning)
Arabic input cost   = Arabic tokens × input price per 1M ÷ 1,000,000
extra vs English    = (Arabic tokens − English tokens) × input price per 1M ÷ 1,000,000

Worked example

The news-style MSA sample in the tool is 316 tokens with the Qwen3.8 27B tokenizer and its English translation is 269. On Qwen3.8 27B at $0.175 per 1M input tokens, sending the Arabic costs $0.000055 against $0.000047 for English: 17% more for the same meaning. Across a million such requests that is $8.22 extra.

How to pay less Arabic tax

  • Pick the model by cost per Arabic request, not by price per token. The table above does that multiplication.
  • Keep system prompts and few-shot examples in English when the model follows them well, and send only user content in Arabic.
  • Strip tashkeel and tatweel (ـ) from text the model only needs to read; normalise Arabic-Indic digits if you send many numbers.
  • Cache repeated instructions: cached input is billed at a fraction of the price. See cached input tokens.

Prices synced 2026-11-14. Google's Gemma 4 26B A4B tokenizer. BroadNet's build is based on this model; we assume it keeps the base tokenizer.

Questions

How many tokens is an Arabic word?

Between 1.8 and 2.4 tokens per word of Modern Standard Arabic in our open dataset, depending on the tokenizer, against 1.3 to 1.3 for English prose. Dialects are similar. Paste your own text above for the exact figure.

Why does Arabic use more tokens than English?

Tokenizers learn their vocabulary from training text, and most training text is English. An Arabic word also carries attached pieces, such as the article ال, the conjunction و and pronoun suffixes, so one written word often maps to several tokens. Bigger multilingual vocabularies learn more whole Arabic words and need fewer tokens.

Which BroadNet model is cheapest for Arabic text?

It depends on both the token count and the price. On Modern Standard Arabic sentences Qwen3.8 27B needs the fewest tokens relative to English (1.31×). Multiply each model's Arabic token count by its price, as the table above does, to compare the real cost. Choosing an LLM for Arabic covers quality too.

Do Arabic diacritics (tashkeel) add tokens?

Yes, a lot. Each harakah is a separate Unicode character, and vocabularies rarely learn vowelled words. In our test the sentence ذهب الولد إلى المدرسة is 5 to 9 tokens across these tokenizers, and the fully vowelled ذَهَبَ الوَلَدُ إِلَى المَدْرَسَةِ is 14 to 27. Strip tashkeel from text the model only needs to read.

Are Arabic-Indic digits counted differently from Western digits?

Yes. ١٢٣٤٥ and 12345 are different characters. In our test 12345 is 2 tokens with DeepSeek, GLM and GPT and 5 with Qwen and Gemma, which split every digit, while ١٢٣٤٥ is 5 to 10 tokens. Normalise digits if you send many numbers.

Is my text sent anywhere?

No. Tokenization runs in your browser. The only download is the public tokenizer file from Hugging Face, once; your text stays on your device.

Go deeper

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity