---
title: "Real LLM API cost by language: Arabic, Hindi, code | BroadRouter"
description: "What the same text costs in English, Arabic, Hindi, Chinese, code and JSON on 108 models and 36 providers, from measured tokenizers. CSV, JSON."
url: https://broadrouter.com/data/real-cost-by-language/
lang: en
updated: 2026-11-15
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Real LLM API cost by language

Updated 2026-11-15

Sending 1M characters of Modern Standard Arabic costs $0.0077 to $6.39, depending on model and provider, against $0.0043 to $4.27 for English. We priced 618 provider rows for 108 models using 16 measured tokenizers. The cheapest Arabic input is Llama 3.1 8B Instruct on DeepInfra. Free CSV and JSON.

[Download CSV](https://broadrouter.com/data/real-cost-by-language.csv) [Download JSON](https://broadrouter.com/data/real-cost-by-language.json) [Croissant metadata](https://broadrouter.com/data/real-cost-by-language.croissant.json) CC BY 4.0 · prices checked 2026-11-14 to 2026-11-15 · BroadNet prices 2026-11-14 · tokens measured 2026-11-15

BroadRouter is BroadNet's own site. BroadNet rows come from its [pricing.json](https://inference.broadnet.ai/pricing.json?utm_source=broadrouter&utm_medium=referral&utm_campaign=data-real-cost&utm_content=note) and are labelled BroadNet; every other price is read from that provider's own page. Where another provider is cheaper, the tables say so.

## Key findings

1.  We priced 614 standard-tier prices from 35 providers, plus 4 BroadNet rows, for 108 models whose tokenizer we measured (16 tokenizers). Each model is mapped to its tokenizer with recorded evidence (the official repo, a byte-identical copy or OpenAI's tiktoken table); BroadNet's Gemma build is assumed to keep Google's tokenizer.
2.  Sending 1M characters of Modern Standard Arabic costs $0.0077 to $6.39 depending on model and provider; English costs $0.0043 to $4.27. The cheapest Arabic input is Llama 3.1 8B Instruct on DeepInfra, and the cheapest English input is Llama 3.1 8B Instruct on DeepInfra.
3.  The cheapest place to send English text is also the cheapest for 12 of the other 14 languages and content kinds. For Hindi and Urdu, the language changes the winner: OpenAI GPT OSS 120B on CoreWeave Forge.
4.  Language tax, in money: on the same 2,009 FLORES-200 sentences, Modern Standard Arabic needs 25% (Ministral 3 tokenizer, 95% interval 24% to 25%) to 203% (Phi-4, 201% to 205%) more tokens than English, so it costs that much more on the same model. For content that is 1M words in English, the median row charges $0.192 more to send it in Arabic and $0.809 more to generate it.
5.  The largest tax in this data is Hindi on the Phi-4 tokenizer: 378% more tokens than the same English sentences (95% interval 374% to 382%). Hindi needs 31% (Gemma 4) to 378% (Phi-4) more; the median row charges $0.644 more to send Hindi content that is 1M words in English.
6.  For Arabic MSA, the lowest price per token (Llama 3.1 8B Instruct on DeepInfra) is also the cheapest per character, at $0.0077 per 1M characters. The tokenizer with the fewest Arabic MSA tokens is Ministral 3, yet its cheapest row (Ministral 3 3B on Amazon Bedrock) costs $0.0298: a low enough per-token price beats a lean tokenizer. Priced per character, 150 pairs of models swap order. Example among the 20 cheapest models per token: Llama 3.2 1B Instruct on Cloudflare Workers AI costs less per token than OpenAI GPT OSS 120B on CoreWeave Forge ($0.0270 against $0.0300), but $0.0104 against $0.0096 per 1M characters of Arabic MSA.
7.  The lowest price per token is Llama 3.1 8B Instruct on DeepInfra ($0.0200 per 1M input tokens), but for Hindi it is not the cheapest: OpenAI GPT OSS 120B on CoreWeave Forge sends 1M characters for $0.0097 against $0.0104, because the gpt-oss o200k\_harmony tokenizer needs 38% fewer tokens for the same Hindi text. The tokenizer with the fewest Hindi tokens is Gemma 4, yet its cheapest row (Gemma 4 26B A4B on DeepInfra) costs $0.0190: a low enough per-token price beats a lean tokenizer. Priced per character, 463 pairs of models swap order. Example among the 20 cheapest models per token: Qwen3-8B on SiliconFlow costs less per token than Gemma 4 26B A4B on DeepInfra ($0.0600 against $0.0700), but $0.0558 against $0.0190 per 1M characters of Hindi.
8.  BroadNet, which runs this site, is not the cheapest input row for any language here. Its cheapest Arabic MSA row is BroadNet Gemma 4 VL 26B at $0.0343 per 1M characters.

## What is the cheapest way to send or generate each language?

Cheapest standard-tier row for 1M characters of each language. Input is what you send (prompts, documents); output is what the model writes. Promotional prices are not ranked.

| Language | Cheapest input | Input per 1M chars | Per 1,000 words | Cheapest output | Output per 1M chars | Dearest input |
| --- | --- | --- | --- | --- | --- | --- |
| **English** | Llama 3.1 8B Instruct on DeepInfra | $0.0043 | $0.000026 | Llama 3.1 8B Instruct on DeepInfra | $0.0087 | $4.27 |
| **Arabic MSA** | Llama 3.1 8B Instruct on DeepInfra | $0.0077 | $0.000046 | Llama 3.1 8B Instruct on DeepInfra | $0.0155 | $6.39 |
| **Gulf Arabic** | Llama 3.1 8B Instruct on DeepInfra | $0.0086 | $0.000048 | Llama 3.1 8B Instruct on DeepInfra | $0.0171 | $7.34 |
| **Egyptian Arabic** | Llama 3.1 8B Instruct on DeepInfra | $0.0079 | $0.000046 | Llama 3.1 8B Instruct on DeepInfra | $0.0157 | $6.66 |
| **Levantine Arabic** | Llama 3.1 8B Instruct on DeepInfra | $0.0080 | $0.000048 | Llama 3.1 8B Instruct on DeepInfra | $0.0160 | $6.71 |
| **Najdi Arabic** | Llama 3.1 8B Instruct on DeepInfra | $0.0078 | $0.000046 | Llama 3.1 8B Instruct on DeepInfra | $0.0155 | $6.40 |
| **Mesopotamian Arabic** | Llama 3.1 8B Instruct on DeepInfra | $0.0079 | $0.000048 | Llama 3.1 8B Instruct on DeepInfra | $0.0157 | $6.54 |
| **Arabizi** | Llama 3.1 8B Instruct on DeepInfra | $0.0083 | $0.000045 | Llama 3.1 8B Instruct on DeepInfra | $0.0166 | $7.75 |
| **Hindi** | OpenAI GPT OSS 120B on CoreWeave Forge | $0.0097 | $0.000049 | Llama 3.1 8B Instruct on DeepInfra | $0.0209 | $6.44 |
| **Urdu** | OpenAI GPT OSS 120B on CoreWeave Forge | $0.0102 | $0.000048 | Gemma 3 4B on Amazon Bedrock | $0.0254 | $6.83 |
| **Chinese** | Llama 3.1 8B Instruct on DeepInfra | $0.0160 | $0.000031 | Llama 3.1 8B Instruct on DeepInfra | $0.0320 | $15.43 |
| **French** | Llama 3.1 8B Instruct on DeepInfra | $0.0055 | $0.000034 | Llama 3.1 8B Instruct on DeepInfra | $0.0110 | $4.65 |
| **Spanish** | Llama 3.1 8B Instruct on DeepInfra | $0.0053 | $0.000032 | Llama 3.1 8B Instruct on DeepInfra | $0.0107 | $4.51 |
| **Code** | Llama 3.1 8B Instruct on DeepInfra | $0.0051 | $0.000039 | Llama 3.1 8B Instruct on DeepInfra | $0.0102 | $5.08 |
| **JSON** | Llama 3.1 8B Instruct on DeepInfra | $0.0063 | $0.000078 | Llama 3.1 8B Instruct on DeepInfra | $0.0126 | $6.29 |

## How much is the language tax in money?

Same meaning, different language: the same 2,009 FLORES-200 sentences (dev and devtest) in English and each language, or Tatoeba sentence pairs where FLORES-200 has no set (Gulf Arabic, Arabizi). Extra tokens = tokens in the language ÷ tokens of the English translation, by tokenizer: the lowest and highest tokenizer, each with its 95% bootstrap interval underneath. Extra cost is for content that is 1M words in English, on the same model and provider: the median over 592 rows, then the lowest and highest.

| Language | Sentences | Extra tokens | Median extra, input | Median extra, output | Lowest and highest extra (input) |
| --- | --- | --- | --- | --- | --- |
| **Arabic MSA** | 2,009 FLORES-200 | 25% to 203% (95% interval 24% to 25% · 201% to 205%) | $0.192 | $0.809 | $0.0139 (GPT OSS 20B on CoreWeave Forge) to $9.27 (o3 Pro on OpenAI) |
| **Gulf Arabic** | 49 Tatoeba | 15% to 154% (95% interval 8% to 23% · 137% to 171%) | $0.137 | $0.615 | $0.0098 (NVIDIA Nemotron 3 Nano 30B A3B on Novita AI) to $7.69 (o3 Pro on OpenAI) |
| **Egyptian Arabic** | 2,009 FLORES-200 | 29% to 195% (95% interval 28% to 30% · 193% to 198%) | $0.199 | $0.872 | $0.0148 (GPT OSS 20B on CoreWeave Forge) to $9.90 (o3 Pro on OpenAI) |
| **Levantine Arabic** | 2,009 FLORES-200 | 24% to 182% (95% interval 23% to 24% · 180% to 184%) | $0.179 | $0.775 | $0.0128 (GPT OSS 20B on CoreWeave Forge) to $8.53 (o3 Pro on OpenAI) |
| **Najdi Arabic** | 2,009 FLORES-200 | 25% to 203% (95% interval 24% to 26% · 201% to 206%) | $0.195 | $0.825 | $0.0141 (GPT OSS 20B on CoreWeave Forge) to $9.38 (o3 Pro on OpenAI) |
| **Mesopotamian Arabic** | 2,009 FLORES-200 | 26% to 198% (95% interval 25% to 27% · 196% to 200%) | $0.195 | $0.841 | $0.0140 (GPT OSS 20B on CoreWeave Forge) to $9.37 (o3 Pro on OpenAI) |
| **Arabizi** | 39 Tatoeba | 30% to 55% (95% interval 21% to 39% · 44% to 66%) | $0.183 | $0.752 | $0.0122 (Llama 3.1 8B Instruct on DeepInfra) to $10.69 (o3 Pro on OpenAI) |
| **Hindi** | 2,009 FLORES-200 | 31% to 378% (95% interval 31% to 32% · 374% to 382%) | $0.644 | $2.65 | $0.0154 (Gemma 3 4B on Amazon Bedrock) to $13.96 (o3 Pro on OpenAI) |
| **Urdu** | 2,009 FLORES-200 | 53% to 336% (95% interval 52% to 54% · 333% to 340%) | $0.503 | $2.29 | $0.0240 (GPT OSS 20B on CoreWeave Forge) to $15.99 (o3 Pro on OpenAI) |
| **Chinese** | 2,009 FLORES-200 | −12% to 88% (95% interval −13% to −11% · 86% to 90%) | −$0.0149 | −$0.0575 | −$0.549 (Kimi K3 on Venice) to $6.24 (o3 Pro on OpenAI) |
| **French** | 2,009 FLORES-200 | 29% to 72% (95% interval 28% to 30% · 71% to 74%) | $0.182 | $0.787 | $0.0130 (GPT OSS 20B on CoreWeave Forge) to $8.69 (o3 Pro on OpenAI) |
| **Spanish** | 2,009 FLORES-200 | 25% to 69% (95% interval 24% to 26% · 68% to 71%) | $0.163 | $0.726 | $0.0115 (GPT OSS 20B on CoreWeave Forge) to $7.68 (o3 Pro on OpenAI) |

A negative figure means the language is cheaper than English on that tokenizer (Chinese packs more meaning per token on some). Per 1M English words of content. The Gulf Arabic and Arabizi sets are small, hence their wide intervals. Najdi Arabic in FLORES-200 is mostly identical to the MSA sentences, so it lands near MSA. Every tokenizer's ratio and interval is in the [tokenizer efficiency dataset](https://broadrouter.com/data/tokenizer-efficiency/).

## When is the token-cheapest model not the money-cheapest?

For Arabic MSA, the lowest price per token (Llama 3.1 8B Instruct on DeepInfra) is also the cheapest per character, at $0.0077 per 1M characters. The tokenizer with the fewest Arabic MSA tokens is Ministral 3, yet its cheapest row (Ministral 3 3B on Amazon Bedrock) costs $0.0298: a low enough per-token price beats a lean tokenizer. Priced per character, 150 pairs of models swap order. Example among the 20 cheapest models per token: Llama 3.2 1B Instruct on Cloudflare Workers AI costs less per token than OpenAI GPT OSS 120B on CoreWeave Forge ($0.0270 against $0.0300), but $0.0104 against $0.0096 per 1M characters of Arabic MSA.

The lowest price per token is Llama 3.1 8B Instruct on DeepInfra ($0.0200 per 1M input tokens), but for Hindi it is not the cheapest: OpenAI GPT OSS 120B on CoreWeave Forge sends 1M characters for $0.0097 against $0.0104, because the gpt-oss o200k\_harmony tokenizer needs 38% fewer tokens for the same Hindi text. The tokenizer with the fewest Hindi tokens is Gemma 4, yet its cheapest row (Gemma 4 26B A4B on DeepInfra) costs $0.0190: a low enough per-token price beats a lean tokenizer. Priced per character, 463 pairs of models swap order. Example among the 20 cheapest models per token: Qwen3-8B on SiliconFlow costs less per token than Gemma 4 26B A4B on DeepInfra ($0.0600 against $0.0700), but $0.0558 against $0.0190 per 1M characters of Hindi.

"Swap order" counts pairs of models, each at its cheapest standard input price, where one is cheaper per token but dearer per character of that language (108 models compared).

## Which model families are covered?

One row per measured tokenizer. Cheapest input per 1M characters, any provider.

| Tokenizer | Models | Price rows | English | Arabic MSA | Hindi |
| --- | --- | --- | --- | --- | --- |
| **DeepSeek V3/V4** · DeepSeek | 12 | 113 | $0.0129 DeepSeek V4 Flash 0731 on DeepInfra | $0.0226 DeepSeek V4 Flash 0731 on DeepInfra | $0.0347 DeepSeek V4 Flash 0731 on DeepInfra |
| **GLM 5** · Z.ai | 5 | 86 | $0.0260 GLM 5.3 Flash on CoreWeave Forge | $0.0496 GLM 5.3 Flash on CoreWeave Forge | $0.107 GLM 5.3 Flash on CoreWeave Forge |
| **Qwen3.5 to 3.8** · Qwen | 9 | 68 | $0.0087 Qwen3.8 27B on Chutes | $0.0121 Qwen3.8 27B on Chutes | $0.0165 Qwen3.8 27B on Chutes |
| **Gemma 4** · Google | 3 | 24 | $0.0153 Gemma 4 26B A4B on DeepInfra | $0.0240 Gemma 4 26B A4B on DeepInfra | $0.0190 Gemma 4 26B A4B on DeepInfra |
| **OpenAI o200k\_base** · OpenAI | 22 | 37 | $0.0107 GPT-5 Nano on Microsoft Azure | $0.0160 GPT-5 Nano on Microsoft Azure | $0.0161 GPT-5 Nano on Microsoft Azure |
| **gpt-oss o200k\_harmony** · OpenAI | 4 | 35 | $0.0064 OpenAI GPT OSS 120B on CoreWeave Forge | $0.0096 OpenAI GPT OSS 120B on CoreWeave Forge | $0.0097 OpenAI GPT OSS 120B on CoreWeave Forge |
| **Llama 3** · Meta | 7 | 35 | $0.0043 Llama 3.1 8B Instruct on DeepInfra | $0.0077 Llama 3.1 8B Instruct on DeepInfra | $0.0104 Llama 3.1 8B Instruct on DeepInfra |
| **Llama 4** · Meta | 2 | 13 | $0.0170 Llama 4 Scout 17B 16E Instruct on GMI Cloud | $0.0296 Llama 4 Scout 17B 16E Instruct on GMI Cloud | $0.0272 Llama 4 Scout 17B 16E Instruct on GMI Cloud |
| **Kimi** · Moonshot AI | 8 | 74 | $0.0848 Kimi K2.6 on Chutes | $0.187 Kimi K2.6 on Chutes | $0.221 Kimi K2.6 on Chutes |
| **MiniMax M2/M3** · MiniMax | 5 | 32 | $0.0314 MiniMax M3.1 Flash on DigitalOcean | $0.0464 MiniMax M3.1 Flash on DigitalOcean | $0.0890 MiniMax M3.1 Flash on DigitalOcean |
| **Gemma 3** · Google | 3 | 13 | $0.0087 Gemma 3 4B on Amazon Bedrock | $0.0137 Gemma 3 4B on Amazon Bedrock | $0.0109 Gemma 3 4B on Amazon Bedrock |
| **Phi-4** · Microsoft | 1 | 3 | $0.0152 Phi-4 on DeepInfra | $0.0495 Phi-4 on DeepInfra | $0.0691 Phi-4 on DeepInfra |
| **Mistral Small 3.1** · Mistral AI | 2 | 6 | $0.0109 Mistral-Small-24B-Instruct-2501 on DeepInfra | $0.0152 Mistral-Small-24B-Instruct-2501 on DeepInfra | $0.0293 Mistral-Small-24B-Instruct-2501 on DeepInfra |
| **Ministral 3** · Mistral AI | 4 | 8 | $0.0219 Ministral 3 3B on Amazon Bedrock | $0.0298 Ministral 3 3B on Amazon Bedrock | $0.0381 Ministral 3 3B on Amazon Bedrock |
| **Nemotron 3** · NVIDIA | 4 | 20 | $0.0109 NVIDIA Nemotron 3.5 Lightning 30B A3B on Crusoe Cloud | $0.0149 NVIDIA Nemotron 3.5 Lightning 30B A3B on Crusoe Cloud | $0.0190 NVIDIA Nemotron 3.5 Lightning 30B A3B on Crusoe Cloud |
| **Qwen3** · Qwen | 17 | 51 | $0.0112 Qwen3 30B A3B (fp8) on Cloudflare Workers AI | $0.0197 Qwen3 30B A3B (fp8) on Cloudflare Workers AI | $0.0475 Qwen3 30B A3B (fp8) on Cloudflare Workers AI |

## What does each model cost per language?

One row per model at its cheapest standard input price, plus every BroadNet row (labelled). Input cost per 1M characters for English, Arabic MSA, Hindi, Chinese and code; "Per 1M tokens" is the input price. Every provider's row, all 13 languages and content kinds, output prices and cost per 1,000 words are in the CSV and JSON.

### **DeepSeek V3/V4 tokenizer** · 12 models, 113 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [DeepSeek V4 Flash 0731](https://broadrouter.com/llm-api-pricing/deepseek-v4-flash-0731/) · DeepInfra | $0.0600 | $0.0129 | $0.0226 | $0.0347 | $0.0352 | $0.0158 |
| [DeepSeek V4 Flash](https://broadrouter.com/llm-api-pricing/deepseek-v4-flash/) · DeepInfra | $0.0900 | $0.0193 | $0.0339 | $0.0520 | $0.0528 | $0.0238 |
| [DeepSeek V4.1 Flash](https://broadrouter.com/llm-api-pricing/deepseek-v4.1-flash/) · CoreWeave Forge | $0.130 | $0.0279 | $0.0489 | $0.0751 | $0.0762 | $0.0343 |
| [DeepSeek V4.1 Flash](https://broadrouter.com/llm-api-pricing/deepseek-v4.1-flash/) · [BroadNet](https://inference.broadnet.ai/pricing.json?utm_source=broadrouter&utm_medium=referral&utm_campaign=data-real-cost&utm_content=row) BroadNet | $0.175 | $0.0375 | $0.0658 | $0.101 | $0.103 | $0.0462 |
| [DeepSeek V4.1 Flash 1017](https://broadrouter.com/llm-api-pricing/deepseek-v4.1-flash-1017/) · Featherless | $0.225 | $0.0482 | $0.0846 | $0.130 | $0.132 | $0.0594 |
| [DeepSeek V3.1](https://broadrouter.com/llm-api-pricing/deepseek-chat-v3.1/) · DeepInfra | $0.250 | $0.0536 | $0.0940 | $0.144 | $0.147 | $0.0660 |
| [DeepSeek V3.2](https://broadrouter.com/llm-api-pricing/deepseek-v3.2/) · DeepInfra | $0.260 | $0.0557 | $0.0978 | $0.150 | $0.152 | $0.0686 |
| [DeepSeek-V3.1-Terminus](https://broadrouter.com/llm-api-pricing/deepseek-v3.1-terminus/) · GMI Cloud | $0.270 | $0.0578 | $0.102 | $0.156 | $0.158 | $0.0713 |
| [DeepSeek-V3.2-Exp](https://broadrouter.com/llm-api-pricing/deepseek-v3.2-exp/) · GMI Cloud | $0.270 | $0.0578 | $0.102 | $0.156 | $0.158 | $0.0713 |
| [DeepSeek R1 0528](https://broadrouter.com/llm-api-pricing/deepseek-r1-0528/) · Featherless | $0.385 | $0.0825 | $0.145 | $0.222 | $0.226 | $0.102 |
| [DeepSeek-V4-Flash-Vision-Exp](https://broadrouter.com/llm-api-pricing/deepseek-v4-flash-vision-exp/) · GMI Cloud | $0.440 | $0.0943 | $0.165 | $0.254 | $0.258 | $0.116 |
| [DeepSeek V4 Pro 0813](https://broadrouter.com/llm-api-pricing/deepseek-v4-pro-0813/) · Chutes | $0.850 | $0.182 | $0.320 | $0.491 | $0.498 | $0.224 |
| [DeepSeek V4 Pro](https://broadrouter.com/llm-api-pricing/deepseek-v4-pro/) · DeepInfra | $1.30 | $0.279 | $0.489 | $0.751 | $0.762 | $0.343 |

### **GLM 5 tokenizer** · 5 models, 86 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [GLM 5.3 Flash](https://broadrouter.com/llm-api-pricing/glm-5.3-flash/) · CoreWeave Forge | $0.120 | $0.0260 | $0.0496 | $0.107 | $0.0712 | $0.0305 |
| [GLM 5.2](https://broadrouter.com/llm-api-pricing/glm-5.2/) · CoreWeave Forge | $0.620 | $0.135 | $0.256 | $0.555 | $0.368 | $0.158 |
| [GLM 5.3](https://broadrouter.com/llm-api-pricing/glm-5.3/) · [BroadNet](https://inference.broadnet.ai/pricing.json?utm_source=broadrouter&utm_medium=referral&utm_campaign=data-real-cost&utm_content=row) BroadNet | $0.780 | $0.169 | $0.322 | $0.698 | $0.463 | $0.198 |
| [GLM 5.1](https://broadrouter.com/llm-api-pricing/glm-5.1/) · Chutes | $0.900 | $0.195 | $0.372 | $0.805 | $0.534 | $0.229 |
| [GLM 5](https://broadrouter.com/llm-api-pricing/glm-5/) · Featherless | $0.950 | $0.206 | $0.393 | $0.850 | $0.564 | $0.242 |
| [GLM 5.3](https://broadrouter.com/llm-api-pricing/glm-5.3/) · Chutes | $1.23 | $0.266 | $0.506 | $1.10 | $0.727 | $0.312 |

### **Qwen3.5 to 3.8 tokenizer** · 9 models, 68 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [Qwen3.8 27B](https://broadrouter.com/llm-api-pricing/qwen3.8-27b/) · Chutes | $0.0390 | $0.0087 | $0.0121 | $0.0165 | $0.0226 | $0.0109 |
| [Qwen3.6 35B A3B](https://broadrouter.com/llm-api-pricing/qwen3.6-35b-a3b/) · CoreWeave Forge | $0.100 | $0.0222 | $0.0310 | $0.0423 | $0.0579 | $0.0278 |
| [Qwen3.5 9B](https://broadrouter.com/llm-api-pricing/qwen3.5-9b/) · DeepInfra | $0.100 | $0.0222 | $0.0310 | $0.0423 | $0.0579 | $0.0278 |
| [Qwen3.5 35B A3B](https://broadrouter.com/llm-api-pricing/qwen3.5-35b-a3b/) · DeepInfra | $0.140 | $0.0311 | $0.0434 | $0.0592 | $0.0810 | $0.0390 |
| [Qwen3.8 27B](https://broadrouter.com/llm-api-pricing/qwen3.8-27b/) · [BroadNet](https://inference.broadnet.ai/pricing.json?utm_source=broadrouter&utm_medium=referral&utm_campaign=data-real-cost&utm_content=row) BroadNet | $0.175 | $0.0388 | $0.0542 | $0.0740 | $0.101 | $0.0487 |
| [Qwen3.5 27B](https://broadrouter.com/llm-api-pricing/qwen3.5-27b/) · Chutes | $0.200 | $0.0444 | $0.0620 | $0.0846 | $0.116 | $0.0556 |
| [Qwen3.6 27B](https://broadrouter.com/llm-api-pricing/qwen3.6-27b/) · Chutes | $0.250 | $0.0555 | $0.0775 | $0.106 | $0.145 | $0.0696 |
| [Qwen3.5 122B A10B](https://broadrouter.com/llm-api-pricing/qwen3.5-122b-a10b/) · SiliconFlow | $0.260 | $0.0577 | $0.0806 | $0.110 | $0.150 | $0.0723 |
| [Qwen3.5 397B A17B](https://broadrouter.com/llm-api-pricing/qwen3.5-397b-a17b/) · Chutes | $0.450 | $0.0999 | $0.139 | $0.190 | $0.260 | $0.125 |
| [Qwen3.8 2.4T A95B](https://broadrouter.com/llm-api-pricing/qwen3.8-2.4t-a95b/) · Featherless | $1.00 | $0.222 | $0.310 | $0.423 | $0.579 | $0.278 |

### **Gemma 4 tokenizer** · 3 models, 24 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [Gemma 4 26B A4B](https://broadrouter.com/llm-api-pricing/gemma-4-26b-a4b-it/) · DeepInfra | $0.0700 | $0.0153 | $0.0240 | $0.0190 | $0.0472 | $0.0204 |
| BroadNet Gemma 4 VL 26B · [BroadNet](https://inference.broadnet.ai/pricing.json?utm_source=broadrouter&utm_medium=referral&utm_campaign=data-real-cost&utm_content=row) BroadNet (tokenizer assumed: Google's base model) | $0.100 | $0.0219 | $0.0343 | $0.0272 | $0.0674 | $0.0291 |
| [Gemma 4 31B](https://broadrouter.com/llm-api-pricing/gemma-4-31b-it/) · CoreWeave Forge | $0.100 | $0.0219 | $0.0343 | $0.0272 | $0.0674 | $0.0291 |

### **OpenAI o200k\_base tokenizer** · 22 models, 37 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [GPT-5 Nano](https://broadrouter.com/llm-api-pricing/gpt-5-nano/) · Microsoft Azure | $0.0500 | $0.0107 | $0.0160 | $0.0161 | $0.0386 | $0.0127 |
| [GPT-4.1 nano (2025-04-14)](https://broadrouter.com/llm-api-pricing/gpt-4.1-nano/) · Microsoft Azure | $0.100 | $0.0213 | $0.0319 | $0.0322 | $0.0771 | $0.0254 |
| [GPT-4o mini](https://broadrouter.com/llm-api-pricing/gpt-4o-mini/) · Microsoft Azure | $0.150 | $0.0320 | $0.0479 | $0.0483 | $0.116 | $0.0381 |
| [GPT-5 Mini](https://broadrouter.com/llm-api-pricing/gpt-5-mini/) · Microsoft Azure | $0.250 | $0.0533 | $0.0798 | $0.0805 | $0.193 | $0.0635 |
| [GPT-4.1 Mini](https://broadrouter.com/llm-api-pricing/gpt-4.1-mini/) · Microsoft Azure | $0.400 | $0.0854 | $0.128 | $0.129 | $0.309 | $0.102 |
| [o1-mini (2024-09-12)](https://broadrouter.com/llm-api-pricing/o1-mini/) · Microsoft Azure | $1.10 | $0.235 | $0.351 | $0.354 | $0.848 | $0.279 |
| [o3-mini (2025-01-31)](https://broadrouter.com/llm-api-pricing/o3-mini/) · Microsoft Azure | $1.10 | $0.235 | $0.351 | $0.354 | $0.848 | $0.279 |
| [o4-mini (2025-04-16)](https://broadrouter.com/llm-api-pricing/o4-mini/) · Microsoft Azure | $1.10 | $0.235 | $0.351 | $0.354 | $0.848 | $0.279 |
| [GPT-5 (2025-08-07)](https://broadrouter.com/llm-api-pricing/gpt-5/) · Microsoft Azure | $1.25 | $0.267 | $0.399 | $0.403 | $0.964 | $0.318 |
| [GPT-5 Chat](https://broadrouter.com/llm-api-pricing/gpt-5-chat/) · Microsoft Azure | $1.25 | $0.267 | $0.399 | $0.403 | $0.964 | $0.318 |
| [GPT-5 Codex](https://broadrouter.com/llm-api-pricing/gpt-5-codex/) · Microsoft Azure | $1.25 | $0.267 | $0.399 | $0.403 | $0.964 | $0.318 |
| [GPT-5 Search API](https://broadrouter.com/llm-api-pricing/gpt-5-search-api/) · OpenAI | $1.25 | $0.267 | $0.399 | $0.403 | $0.964 | $0.318 |
| [GPT-4.1 (2025-04-14)](https://broadrouter.com/llm-api-pricing/gpt-4.1/) · Microsoft Azure | $2.00 | $0.427 | $0.639 | $0.644 | $1.54 | $0.508 |
| [o3 (2025-04-16)](https://broadrouter.com/llm-api-pricing/o3/) · Microsoft Azure | $2.00 | $0.427 | $0.639 | $0.644 | $1.54 | $0.508 |
| [GPT-4o (2024-08-06)](https://broadrouter.com/llm-api-pricing/gpt-4o-2024-08-06/) · Microsoft Azure | $2.50 | $0.533 | $0.798 | $0.805 | $1.93 | $0.635 |
| [GPT-4o (2024-11-20)](https://broadrouter.com/llm-api-pricing/gpt-4o-2024-11-20/) · Microsoft Azure | $2.50 | $0.533 | $0.798 | $0.805 | $1.93 | $0.635 |
| [GPT-4o](https://broadrouter.com/llm-api-pricing/gpt-4o/) · DigitalOcean | $2.50 | $0.533 | $0.798 | $0.805 | $1.93 | $0.635 |
| [GPT-4o (2024-05-13)](https://broadrouter.com/llm-api-pricing/gpt-4o-2024-05-13/) · Microsoft Azure | $5.00 | $1.07 | $1.60 | $1.61 | $3.86 | $1.27 |
| [o3 Deep Research](https://broadrouter.com/llm-api-pricing/o3-deep-research/) · Microsoft Azure | $10.00 | $2.13 | $3.19 | $3.22 | $7.71 | $2.54 |
| [GPT-5 Pro](https://broadrouter.com/llm-api-pricing/gpt-5-pro/) · Microsoft Azure | $15.00 | $3.20 | $4.79 | $4.83 | $11.57 | $3.81 |
| [o1 (2024-12-17)](https://broadrouter.com/llm-api-pricing/o1/) · Microsoft Azure | $15.00 | $3.20 | $4.79 | $4.83 | $11.57 | $3.81 |
| [o3 Pro](https://broadrouter.com/llm-api-pricing/o3-pro/) · OpenAI | $20.00 | $4.27 | $6.39 | $6.44 | $15.43 | $5.08 |

### **gpt-oss o200k\_harmony tokenizer** · 4 models, 35 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [OpenAI GPT OSS 120B](https://broadrouter.com/llm-api-pricing/gpt-oss-120b/) · CoreWeave Forge | $0.0300 | $0.0064 | $0.0096 | $0.0097 | $0.0231 | $0.0076 |
| [GPT OSS 20B](https://broadrouter.com/llm-api-pricing/gpt-oss-20b/) · CoreWeave Forge | $0.0300 | $0.0064 | $0.0096 | $0.0097 | $0.0231 | $0.0076 |
| [GPT OSS Safeguard 20B](https://broadrouter.com/llm-api-pricing/gpt-oss-safeguard-20b/) · Amazon Bedrock | $0.0700 | $0.0149 | $0.0224 | $0.0225 | $0.0540 | $0.0178 |
| [GPT OSS Safeguard 120B](https://broadrouter.com/llm-api-pricing/gpt-oss-safeguard-120b/) · Amazon Bedrock | $0.150 | $0.0320 | $0.0479 | $0.0483 | $0.116 | $0.0381 |

### **Llama 3 tokenizer** · 7 models, 35 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [Llama 3.1 8B Instruct](https://broadrouter.com/llm-api-pricing/llama-3.1-8b-instruct/) · DeepInfra | $0.0200 | $0.0043 | $0.0077 | $0.0104 | $0.0160 | $0.0051 |
| [Llama 3.2 1B Instruct](https://broadrouter.com/llm-api-pricing/llama-3.2-1b-instruct/) · Cloudflare Workers AI | $0.0270 | $0.0059 | $0.0104 | $0.0141 | $0.0216 | $0.0069 |
| [Llama 3.2 3B Instruct](https://broadrouter.com/llm-api-pricing/llama-3.2-3b-instruct/) · Featherless | $0.0509 | $0.0110 | $0.0197 | $0.0266 | $0.0407 | $0.0130 |
| [Llama 3 8B Instruct](https://broadrouter.com/llm-api-pricing/llama-3-8b-instruct/) · Featherless | $0.0925 | $0.0201 | $0.0358 | $0.0483 | $0.0740 | $0.0235 |
| [Llama 3.3 70B](https://broadrouter.com/llm-api-pricing/llama-3.3-70b-instruct/) · DeepInfra | $0.100 | $0.0217 | $0.0387 | $0.0522 | $0.0800 | $0.0254 |
| [Llama 3.1 Instruct (70B)](https://broadrouter.com/llm-api-pricing/llama-3.1-70b-instruct/) · Cloudflare Workers AI | $0.293 | $0.0635 | $0.113 | $0.153 | $0.234 | $0.0746 |
| [Llama 3 Instruct (70B)](https://broadrouter.com/llm-api-pricing/llama-3-70b-instruct/) · Amazon Bedrock | $2.65 | $0.575 | $1.02 | $1.38 | $2.12 | $0.674 |

### **Llama 4 tokenizer** · 2 models, 13 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [Llama 4 Scout 17B 16E Instruct](https://broadrouter.com/llm-api-pricing/llama-4-scout/) · GMI Cloud | $0.0800 | $0.0170 | $0.0296 | $0.0272 | $0.0530 | $0.0204 |
| [Llama 4 Maverick 17B 128E Instruct FP8](https://broadrouter.com/llm-api-pricing/llama-4-maverick/) · Amazon Bedrock | $0.240 | $0.0510 | $0.0889 | $0.0815 | $0.159 | $0.0613 |

### **Kimi tokenizer** · 8 models, 74 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [Kimi K2.6](https://broadrouter.com/llm-api-pricing/kimi-k2.6/) · Chutes | $0.390 | $0.0848 | $0.187 | $0.221 | $0.213 | $0.101 |
| [Kimi K2.5](https://broadrouter.com/llm-api-pricing/kimi-k2.5/) · SiliconFlow | $0.450 | $0.0978 | $0.216 | $0.255 | $0.245 | $0.117 |
| [Kimi K2 Instruct](https://broadrouter.com/llm-api-pricing/kimi-k2/) · Novita AI | $0.570 | $0.124 | $0.273 | $0.324 | $0.311 | $0.148 |
| [Kimi K2 Thinking](https://broadrouter.com/llm-api-pricing/kimi-k2-thinking/) · Amazon Bedrock | $0.600 | $0.130 | $0.288 | $0.341 | $0.327 | $0.156 |
| [Kimi K2.7 Code](https://broadrouter.com/llm-api-pricing/kimi-k2.7-code/) · CoreWeave Forge | $0.600 | $0.130 | $0.288 | $0.341 | $0.327 | $0.156 |
| [Kimi K2 Instruct 0905](https://broadrouter.com/llm-api-pricing/kimi-k2-0905/) · Featherless | $0.600 | $0.130 | $0.288 | $0.341 | $0.327 | $0.156 |
| [Kimi K3](https://broadrouter.com/llm-api-pricing/kimi-k3/) · Chutes | $1.85 | $0.402 | $0.887 | $1.05 | $1.01 | $0.480 |
| [Kimi K3.1 Pro](https://broadrouter.com/llm-api-pricing/kimi-k3.1-pro/) · Alibaba Cloud | $2.00 | $0.435 | $0.959 | $1.14 | $1.09 | $0.519 |

### **MiniMax M2/M3 tokenizer** · 5 models, 32 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [MiniMax M3.1 Flash](https://broadrouter.com/llm-api-pricing/minimax-m3.1-flash/) · DigitalOcean | $0.150 | $0.0314 | $0.0464 | $0.0890 | $0.0837 | $0.0387 |
| [MiniMax M3](https://broadrouter.com/llm-api-pricing/minimax-m3/) · CoreWeave Forge | $0.230 | $0.0482 | $0.0712 | $0.137 | $0.128 | $0.0593 |
| [MiniMax M2.5](https://broadrouter.com/llm-api-pricing/minimax-m2.5/) · Venice | $0.270 | $0.0565 | $0.0836 | $0.160 | $0.151 | $0.0696 |
| [MiniMax M2](https://broadrouter.com/llm-api-pricing/minimax-m2/) · Featherless | $0.278 | $0.0581 | $0.0859 | $0.165 | $0.155 | $0.0716 |
| [Minimax M2.1](https://broadrouter.com/llm-api-pricing/minimax-m2.1/) · Amazon Bedrock | $0.300 | $0.0628 | $0.0929 | $0.178 | $0.167 | $0.0774 |

### **Gemma 3 tokenizer** · 3 models, 13 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [Gemma 3 4B](https://broadrouter.com/llm-api-pricing/gemma-3-4b-it/) · Amazon Bedrock | $0.0400 | $0.0087 | $0.0137 | $0.0109 | $0.0270 | $0.0117 |
| [Gemma 3 12B](https://broadrouter.com/llm-api-pricing/gemma-3-12b-it/) · DeepInfra | $0.0500 | $0.0109 | $0.0172 | $0.0136 | $0.0337 | $0.0146 |
| [Gemma 3 27B](https://broadrouter.com/llm-api-pricing/gemma-3-27b-it/) · DeepInfra | $0.0800 | $0.0175 | $0.0275 | $0.0217 | $0.0539 | $0.0233 |

### **Phi-4 tokenizer** · 1 models, 3 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [Phi-4](https://broadrouter.com/llm-api-pricing/phi-4/) · DeepInfra | $0.0700 | $0.0152 | $0.0495 | $0.0691 | $0.0815 | $0.0178 |

### **Mistral Small 3.1 tokenizer** · 2 models, 6 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [Mistral-Small-24B-Instruct-2501](https://broadrouter.com/llm-api-pricing/mistral-small-24b-instruct-2501/) · DeepInfra | $0.0500 | $0.0109 | $0.0152 | $0.0293 | $0.0451 | $0.0136 |
| [Mistral Small 3.1 24B Instruct](https://broadrouter.com/llm-api-pricing/mistral-small-3.1-24b-instruct/) · Google Vertex AI | $0.100 | $0.0219 | $0.0303 | $0.0585 | $0.0901 | $0.0271 |

### **Ministral 3 tokenizer** · 4 models, 8 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [Ministral 3 3B](https://broadrouter.com/llm-api-pricing/ministral-3b-2512/) · Amazon Bedrock | $0.100 | $0.0219 | $0.0298 | $0.0381 | $0.0901 | $0.0269 |
| [Ministral 3 8B](https://broadrouter.com/llm-api-pricing/ministral-8b-2512/) · Amazon Bedrock | $0.150 | $0.0328 | $0.0447 | $0.0571 | $0.135 | $0.0404 |
| [Ministral 3 14B](https://broadrouter.com/llm-api-pricing/ministral-14b-2512/) · Amazon Bedrock | $0.200 | $0.0438 | $0.0596 | $0.0761 | $0.180 | $0.0539 |
| [Devstral 2 123B](https://broadrouter.com/llm-api-pricing/devstral-2512/) · Amazon Bedrock | $0.400 | $0.0876 | $0.119 | $0.152 | $0.360 | $0.108 |

### **Nemotron 3 tokenizer** · 4 models, 20 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [NVIDIA Nemotron 3.5 Lightning 30B A3B](https://broadrouter.com/llm-api-pricing/nemotron-3.5-lightning/) · Crusoe Cloud | $0.0500 | $0.0109 | $0.0149 | $0.0190 | $0.0451 | $0.0135 |
| [NVIDIA Nemotron 3 Nano 30B A3B](https://broadrouter.com/llm-api-pricing/nemotron-3-nano-30b-a3b/) · Novita AI | $0.0500 | $0.0109 | $0.0149 | $0.0190 | $0.0451 | $0.0135 |
| [NVIDIA Nemotron 3 Super 120B A12B](https://broadrouter.com/llm-api-pricing/nemotron-3-super-120b-a12b/) · DeepInfra | $0.0750 | $0.0164 | $0.0224 | $0.0286 | $0.0676 | $0.0202 |
| [NVIDIA Nemotron 3 Ultra](https://broadrouter.com/llm-api-pricing/nemotron-3-ultra-550b-a55b/) · CoreWeave Forge | $0.500 | $0.109 | $0.149 | $0.190 | $0.451 | $0.135 |

### **Qwen3 tokenizer** · 17 models, 51 provider rows

| Model · cheapest provider | Input per 1M tokens | English | Arabic MSA | Hindi | Chinese | Code |
| --- | --- | --- | --- | --- | --- | --- |
| [Qwen3 30B A3B (fp8)](https://broadrouter.com/llm-api-pricing/qwen3-30b-a3b/) · Cloudflare Workers AI | $0.0510 | $0.0112 | $0.0197 | $0.0475 | $0.0326 | $0.0133 |
| [Qwen3-8B](https://broadrouter.com/llm-api-pricing/qwen3-8b/) · SiliconFlow | $0.0600 | $0.0132 | $0.0232 | $0.0558 | $0.0384 | $0.0156 |
| [Qwen3-14B](https://broadrouter.com/llm-api-pricing/qwen3-14b/) · SiliconFlow | $0.0700 | $0.0154 | $0.0271 | $0.0651 | $0.0448 | $0.0182 |
| [Qwen3 Coder 30B A3B](https://broadrouter.com/llm-api-pricing/qwen3-coder-30b-a3b-instruct/) · SiliconFlow | $0.0700 | $0.0154 | $0.0271 | $0.0651 | $0.0448 | $0.0182 |
| [Qwen3 32B FP8](https://broadrouter.com/llm-api-pricing/qwen3-32b/) · DeepInfra | $0.0800 | $0.0176 | $0.0310 | $0.0745 | $0.0512 | $0.0208 |
| [Qwen3 235B A22B Instruct 2507 FP8](https://broadrouter.com/llm-api-pricing/qwen3-235b-a22b-2507/) · DeepInfra | $0.0900 | $0.0198 | $0.0348 | $0.0838 | $0.0576 | $0.0234 |
| [Qwen3 Next 80B A3B Instruct](https://broadrouter.com/llm-api-pricing/qwen3-next-80b-a3b-instruct/) · DeepInfra | $0.0900 | $0.0198 | $0.0348 | $0.0838 | $0.0576 | $0.0234 |
| [Qwen3-30B-A3B-Instruct-2507](https://broadrouter.com/llm-api-pricing/qwen3-30b-a3b-instruct-2507/) · SiliconFlow | $0.0900 | $0.0198 | $0.0348 | $0.0838 | $0.0576 | $0.0234 |
| [Qwen3 Coder Next](https://broadrouter.com/llm-api-pricing/qwen3-coder-next/) · Parasail | $0.120 | $0.0264 | $0.0464 | $0.112 | $0.0768 | $0.0312 |
| [Qwen3 VL 30B A3B Instruct](https://broadrouter.com/llm-api-pricing/qwen3-vl-30b-a3b-instruct/) · DeepInfra | $0.150 | $0.0329 | $0.0581 | $0.140 | $0.0960 | $0.0390 |
| [Qwen3 Next 80B A3B Thinking](https://broadrouter.com/llm-api-pricing/qwen3-next-80b-a3b-thinking/) · GMI Cloud | $0.150 | $0.0329 | $0.0581 | $0.140 | $0.0960 | $0.0390 |
| [Qwen3-VL 8B](https://broadrouter.com/llm-api-pricing/qwen3-vl-8b-instruct/) · SiliconFlow | $0.180 | $0.0395 | $0.0697 | $0.168 | $0.115 | $0.0468 |
| [Qwen3 VL 235B A22B](https://broadrouter.com/llm-api-pricing/qwen3-vl-235b-a22b-instruct/) · DeepInfra | $0.200 | $0.0439 | $0.0774 | $0.186 | $0.128 | $0.0520 |
| [Qwen3-VL-32B-Instruct](https://broadrouter.com/llm-api-pricing/qwen3-vl-32b-instruct/) · SiliconFlow | $0.200 | $0.0439 | $0.0774 | $0.186 | $0.128 | $0.0520 |
| [Qwen3-VL-32B-Thinking](https://broadrouter.com/llm-api-pricing/qwen3-vl-32b-thinking/) · SiliconFlow | $0.200 | $0.0439 | $0.0774 | $0.186 | $0.128 | $0.0520 |
| [Qwen3 Coder 480B A35B](https://broadrouter.com/llm-api-pricing/qwen3-coder/) · Google Vertex AI | $0.220 | $0.0483 | $0.0851 | $0.205 | $0.141 | $0.0572 |
| [Qwen3-VL-30B-A3B-Thinking](https://broadrouter.com/llm-api-pricing/qwen3-vl-30b-a3b-thinking/) · SiliconFlow | $0.290 | $0.0637 | $0.112 | $0.270 | $0.186 | $0.0754 |

## How are models matched to tokenizers?

A script (`scripts/dataset-tokenizer-models.mjs`) opens each model's own Hugging Face repo at its current commit and accepts the match only if the tokenizer file is byte-identical to the measured one, or has the same vocabulary, merges, normalizer and pre-tokenizer and gives identical counts on all 52 texts (including every FLORES-200 sentence). OpenAI's hosted models use the encoding OpenAI's tiktoken library names for them. Checked 2026-11-15: 107 models matched, 14 candidates rejected.

| Tokenizer | Measured file | Models matched, and how |
| --- | --- | --- |
| **DeepSeek V3/V4** | [deepseek-ai/DeepSeek-V4.1-Flash @ 314113f](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/tree/314113f8a7dadb388215396ba5fbee2c610a04d6) | the measured tokenizer itself: deepseek-v4.1-flash. byte-identical tokenizer file: deepseek-v4.1-flash-1017, deepseek-v4-flash-vision-exp. same vocabulary, merges and pre-tokenizer; identical counts on every text: deepseek-v4-flash, deepseek-v4-flash-0731, deepseek-v4-pro, deepseek-v4-pro-0813, deepseek-v3.2, deepseek-v3.2-exp, deepseek-v3.1-terminus, deepseek-chat-v3.1, deepseek-r1-0528. |
| **GLM 5** | [zai-org/GLM-5.3 @ ba32f8c](https://huggingface.co/zai-org/GLM-5.3/tree/ba32f8c19c8738e2c57c71798c601d477a7bf844) | the measured tokenizer itself: glm-5.3. byte-identical tokenizer file: glm-5.3-flash, glm-5.2, glm-5.1, glm-5. |
| **Qwen3.5 to 3.8** | [Qwen/Qwen3.8-27B @ 3756181](https://huggingface.co/Qwen/Qwen3.8-27B/tree/37561816e66b5b9153b1b64c10968c18c1e6f4a0) | the measured tokenizer itself: qwen3.8-27b. byte-identical tokenizer file: qwen3.8-2.4t-a95b. same vocabulary, merges and pre-tokenizer; identical counts on every text: qwen3.6-35b-a3b, qwen3.6-27b, qwen3.5-397b-a17b, qwen3.5-122b-a10b, qwen3.5-35b-a3b, qwen3.5-27b, qwen3.5-9b. |
| **Gemma 4** | [google/gemma-4-26B-A4B-it @ 4d7ae49](https://huggingface.co/google/gemma-4-26B-A4B-it/tree/4d7ae4984b7db7de8f8457170b3f1a419ee76d52) | the measured tokenizer itself: gemma-4-26b-a4b-it. byte-identical tokenizer file: gemma-4-31b-it. |
| **OpenAI o200k\_base** | o200k\_base | OpenAI's tiktoken model table: gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, gpt-4o, gpt-4o-2024-05-13, gpt-4o-2024-08-06, gpt-4o-2024-11-20, gpt-4o-mini, gpt-5, gpt-5-chat, gpt-5-codex, gpt-5-mini, gpt-5-nano, gpt-5-pro, gpt-5-search-api, o1, o1-mini, o3, o3-mini, o3-pro, o3-deep-research, o4-mini. |
| **gpt-oss o200k\_harmony** | [openai/gpt-oss-120b @ b5c939d](https://huggingface.co/openai/gpt-oss-120b/tree/b5c939de8f754692c1647ca79fbf85e8c1e70f8a) | the measured tokenizer itself: gpt-oss-120b. byte-identical tokenizer file: gpt-oss-20b, gpt-oss-safeguard-120b, gpt-oss-safeguard-20b. |
| **Llama 3** | [meta-llama/Llama-3.3-70B-Instruct @ 6f6073b](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct/tree/6f6073b423013f6a7d4d9f39144961bfbfbc386b) | the measured tokenizer itself: llama-3.3-70b-instruct. same vocabulary, merges and pre-tokenizer; identical counts on every text: llama-3.1-8b-instruct, llama-3.1-70b-instruct, llama-3.2-1b-instruct, llama-3.2-3b-instruct, llama-3-8b-instruct, llama-3-70b-instruct. |
| **Llama 4** | [meta-llama/Llama-4-Maverick-17B-128E-Instruct @ 73d1471](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct/tree/73d14711bcc77c16df3470856949c3764056b617) | the measured tokenizer itself: llama-4-maverick. byte-identical tokenizer file: llama-4-scout. |
| **Kimi** | [moonshotai/Kimi-K3 @ f831ab6](https://huggingface.co/moonshotai/Kimi-K3/tree/f831ab66814297da540d832a5235f8e904f29d06) | the measured tokenizer itself: kimi-k3. byte-identical tokenizer file: kimi-k3.1-pro, kimi-k2.7-code, kimi-k2.6, kimi-k2.5, kimi-k2-thinking, kimi-k2-0905, kimi-k2. |
| **MiniMax M2/M3** | [MiniMaxAI/MiniMax-M3 @ d95e30c](https://huggingface.co/MiniMaxAI/MiniMax-M3/tree/d95e30cb912fe3021bb9c2ba66a8ce60b6c937a9) | the measured tokenizer itself: minimax-m3. byte-identical tokenizer file: minimax-m3.1-flash. same vocabulary, merges and pre-tokenizer; identical counts on every text: minimax-m2.5, minimax-m2.1, minimax-m2. |
| **Gemma 3** | [google/gemma-3-27b-it @ 005ad34](https://huggingface.co/google/gemma-3-27b-it/tree/005ad3404e59d6023443cb575daa05336842228a) | the measured tokenizer itself: gemma-3-27b-it. byte-identical tokenizer file: gemma-3-12b-it, gemma-3-4b-it. |
| **Phi-4** | [microsoft/phi-4 @ 2db69c1](https://huggingface.co/microsoft/phi-4/tree/2db69c1c3e91a05d2c64a3185acfbaf36f744e25) | the measured tokenizer itself: phi-4. |
| **Mistral Small 3.1** | [mistralai/Mistral-Small-3.1-24B-Instruct-2503 @ 68faf51](https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/tree/68faf511d618ef198fef186659617cfd2eb8e33a) | the measured tokenizer itself: mistral-small-3.1-24b-instruct. byte-identical tokenizer file: mistral-small-24b-instruct-2501. |
| **Ministral 3** | [mistralai/Ministral-3-14B-Instruct-2512 @ 29439f8](https://huggingface.co/mistralai/Ministral-3-14B-Instruct-2512/tree/29439f81c2be264d8d393273f99e7db9c0961120) | the measured tokenizer itself: ministral-14b-2512. byte-identical tokenizer file: ministral-8b-2512, ministral-3b-2512, devstral-2512. |
| **Nemotron 3** | [nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 @ 77df655](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/tree/77df655d5e9f8362164ed14dd8b48f8bce657498) | the measured tokenizer itself: nemotron-3-ultra-550b-a55b. byte-identical tokenizer file: nemotron-3-super-120b-a12b, nemotron-3.5-lightning. same vocabulary, merges and pre-tokenizer; identical counts on every text: nemotron-3-nano-30b-a3b. |
| **Qwen3** | [Qwen/Qwen3-32B @ 9216db5](https://huggingface.co/Qwen/Qwen3-32B/tree/9216db5781bf21249d130ec9da846c4624c16137) | the measured tokenizer itself: qwen3-32b. byte-identical tokenizer file: qwen3-14b, qwen3-8b, qwen3-30b-a3b, qwen3-30b-a3b-instruct-2507, qwen3-235b-a22b-2507, qwen3-next-80b-a3b-instruct, qwen3-next-80b-a3b-thinking. same vocabulary, merges and pre-tokenizer; identical counts on every text: qwen3-coder, qwen3-coder-30b-a3b-instruct, qwen3-coder-next, qwen3-vl-235b-a22b-instruct, qwen3-vl-30b-a3b-instruct, qwen3-vl-30b-a3b-thinking, qwen3-vl-32b-instruct, qwen3-vl-32b-thinking, qwen3-vl-8b-instruct. |

The repo and commit checked for every model, and the evidence for each match, are in the JSON download (`modelMapping`).

### 14 candidates left out, and why

-   `glm-4.7` (GLM 5): tokenizer.json differs from zai-org/GLM-5.3 in model.
-   `gemma-4-12b-it` (Gemma 4): tokenizer.json differs from google/gemma-4-26B-A4B-it in model.
-   `gemma-4-e2b-it` (Gemma 4): tokenizer.json differs from google/gemma-4-26B-A4B-it in model.
-   `llama-3.2-11b-vision-instruct` (Llama 3): gated, tokenizer.json differs from the measured file, and no byte-identical ungated mirror was found.
-   `llama-3.2-90b-vision-instruct` (Llama 3): gated, tokenizer.json differs from the measured file, and no byte-identical ungated mirror was found.
-   `phi-4-reasoning` (Phi-4): tokenizer.json differs from microsoft/phi-4 in model.
-   `phi-4-reasoning-plus` (Phi-4): tokenizer.json differs from microsoft/phi-4 in model.
-   `mistral-small-3.2-24b-instruct` (Mistral Small 3.1): no tokenizer.json in the repo.
-   `nemotron-3.5-super-vl` (Nemotron 3): tokenizer.json differs from nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 in model.
-   `gpt-5.1` (OpenAI o200k\_base): tiktoken names no encoding for "gpt-5.1" except through the bare "gpt-5" prefix, which its own comment says can match non-existent models.
-   `gpt-5.2` (OpenAI o200k\_base): tiktoken names no encoding for "gpt-5.2" except through the bare "gpt-5" prefix, which its own comment says can match non-existent models.
-   `gpt-5.4` (OpenAI o200k\_base): tiktoken names no encoding for "gpt-5.4" except through the bare "gpt-5" prefix, which its own comment says can match non-existent models.
-   `gpt-5.5` (OpenAI o200k\_base): tiktoken names no encoding for "gpt-5.5" except through the bare "gpt-5" prefix, which its own comment says can match non-existent models.
-   `gpt-5.6-sol` (OpenAI o200k\_base): tiktoken names no encoding for "gpt-5.6-sol" except through the bare "gpt-5" prefix, which its own comment says can match non-existent models.

## Method

-   Token counts: the 16-tokenizer study at /data/tokenizer-efficiency/ (version 4.0, measured 2026-11-15), each tokenizer's official file pinned to a commit. For each language, its texts are pooled (the 2,009 FLORES-200 sentences plus Wikipedia leads where we have them; English is Wikipedia and Project Gutenberg prose), then tokens ÷ characters and tokens ÷ words.
-   Model mapping: a priced model is linked to a measured tokenizer only when a script confirms it in the model's own Hugging Face repo: the same tokenizer file (same Git object ID), or the same vocabulary, merges, normalizer and pre-tokenizer with identical counts on all 52 texts, FLORES-200 sentences included. OpenAI's hosted models use the encoding OpenAI's tiktoken library names for them. Everything else is left out, including API-only models with no open weights.
-   Prices: standard-tier input and output per 1M tokens, read from each provider's own pricing page (the LLM API provider price database), and BroadNet's pricing.json. Batch, flex and priority tiers, free rows and cached-input prices are not used. Promotional prices are listed but not ranked.
-   Cost per 1M characters = tokens per character × price per 1M tokens. Cost per 1,000 words = tokens per word × 1,000 × price ÷ 1M.
-   Language tax: the extra cost of the same content in another language than in English, on the same model and provider, for content that is 1M words in English. It uses parallel sentences: FLORES-200 dev and devtest, 2,009 aligned sentences per language (CC BY-SA 4.0; we publish counts only), and for languages FLORES-200 lacks, Tatoeba sentence pairs (Gulf Arabic 49, Arabizi 39 sentences). Each extra-token figure has a 95% interval from a bootstrap over sentences (10,000 seeded resamples).
-   Limits: counts exclude chat-template and reasoning tokens; reasoning models bill thinking tokens as output. FLORES-200 is translated news and encyclopedia text, and Wikipedia leads are not your traffic. Treat the figures as a planning guide and measure your own text.

BroadNet rows: the four BroadNet chat models with a measured tokenizer, priced from pricing.json. BroadNet Gemma 4 VL 26B is BroadNet's build of Google's Gemma 4 26B; BroadNet has not published its tokenizer, so we assume it keeps Google's, and the row says so.

Version 1.1 (token data v4): the language tax now uses FLORES-200 parallel sentences (2,009 per language) instead of 33 to 73 Tatoeba sentences, adds Najdi and Mesopotamian Arabic, and gives a 95% interval for each extra-token figure. Gulf Arabic and Arabizi still use Tatoeba. Prices and the model mapping are unchanged in method.

Update frequency: costs are recomputed on every build from the latest provider prices and BroadNet's pricing.json; the model mapping is re-checked when the script is re-run; token counts change only when a tokenizer or the corpus changes.

## What do the CSV columns mean?

One row per price row and language (9,270 rows).

| Column | Meaning |
| --- | --- |
| `provider` | Who sells the model. "BroadNet Inference" rows are BroadNet, the owner of this site. |
| `provider_id` | Provider ID, as in the LLM API provider price database. |
| `model` | Model name as the provider lists it. |
| `model_slug` | Model ID in our price database; /llm-api-pricing/<model\_slug>/ has every provider. |
| `tokenizer` | Measured tokenizer this model uses (see tokenizer\_mapping). |
| `tokenizer_mapping` | Why the model maps to that tokenizer: measured, identical (byte-identical file), same-core (same vocabulary, merges and pre-tokenizer, identical counts), tiktoken (OpenAI's model table) or broadnet (BroadNet build of the measured model, assumed to keep its tokenizer). |
| `language` | Language or content kind: en, ar-msa, ar-gulf, ar-egyptian, ar-levantine, ar-najdi, ar-mesopotamian, arabizi, hi, ur, zh, fr, es, code, json. |
| `input_usd_per_1m_tokens` | Standard-tier input price per 1M tokens, from the provider's page. |
| `output_usd_per_1m_tokens` | Standard-tier output price per 1M tokens. |
| `tokens_per_1k_chars` | Measured tokens per 1,000 characters of this language with this tokenizer. |
| `tokens_per_1k_words` | Measured tokens per 1,000 words (Intl.Segmenter words; for Chinese, segmented words). |
| `input_usd_per_1m_chars` | Cost to send 1M characters of this language: tokens per character × input price. |
| `output_usd_per_1m_chars` | Cost to receive 1M characters of this language as output. |
| `input_usd_per_1k_words` | Cost to send 1,000 words. |
| `output_usd_per_1k_words` | Cost to receive 1,000 words. |
| `promo` | yes if the provider marks the price as promotional; such rows are not ranked in the findings. |
| `source_url` | Where the price was read. |
| `checked` | Date the price was checked. |

## How to cite this dataset

Our compilation and measurements (token counts, the model-to-tokenizer mapping, cost per character and per word, findings, tables and method): [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). Free to use, share and adapt with attribution and a link to this page.

Each price is a fact published by the provider named in its row, at the source URL given per row; we do not license those pages to you. Credit the provider as the source of each price. The texts behind the token counts keep their own licences (see the tokenizer efficiency dataset); FLORES-200 sentences (CC BY-SA 4.0) are not included anywhere in our files, only our counts.

**Plain text**

```
BroadRouter (2026). Real cost by language: what the same text costs on each LLM model and provider (version 1.1) [Data set]. BroadNet Technologies. https://broadrouter.com/data/real-cost-by-language/
```

**BibTeX**

```
@misc{broadrouter_real_cost_by_language,
  author       = {BroadRouter},
  title        = {Real cost by language: what the same text costs on each LLM model and provider},
  year         = {2026},
  version      = {1.1},
  publisher    = {BroadNet Technologies},
  howpublished = {\url{https://broadrouter.com/data/real-cost-by-language/}},
  note         = {Data set, data as of 2026-11-15},
}
```

Count and price your own text

-   [Arabic token counter](https://broadrouter.com/tools/arabic-token-counter/) Arabic vs English token counts per tokenizer: the "Arabic tax" in tokens and money.
-   [LLM token counter](https://broadrouter.com/tools/token-counter/) Exact token counts with each model's real tokenizer, with a token view and cost.
-   [LLM API cost calculator](https://broadrouter.com/tools/llm-api-cost-calculator/) Price a workload on every model at once, with cache hits, reasoning and a flat-plan check.

[All free tools](https://broadrouter.com/tools/)

## Questions

### How much does it cost to process Arabic text with an LLM API?

Sending 1M characters of Modern Standard Arabic costs $0.0077 to $6.39 across the 618 price rows here; the cheapest is Llama 3.1 8B Instruct on DeepInfra. Output costs more: the cheapest is Llama 3.1 8B Instruct on DeepInfra at $0.0155 per 1M characters.

### Why does the same text cost more in Arabic or Hindi?

APIs bill per token, and a tokenizer trained on less Arabic or Hindi text splits those words into more pieces. The same sentence can need twice the tokens of its English version on one tokenizer and barely more on another, so the language changes which model is cheapest.

### Which LLM API is cheapest for Hindi?

For input, OpenAI GPT OSS 120B on CoreWeave Forge at $0.0097 per 1M characters of Hindi; for output, Llama 3.1 8B Instruct on DeepInfra at $0.0209. These are list prices for the standard tier; check quality on your own text.

### What is the language tax?

The extra you pay to process the same meaning in another language than in English, because it needs more tokens. Here it is measured on the same 2,009 FLORES-200 sentences in each language: for Modern Standard Arabic, 25% to 203% more tokens depending on the tokenizer.

### Is the model with the lowest price per token the cheapest for every language?

No. Priced per character of Arabic, 150 pairs of models swap order; for Hindi, 463. Compare cost per character or per word in your language, not price per token.

### Can I use this data?

Yes. Our compilation and measurements (token counts, the model-to-tokenizer mapping, cost per character and per word, findings, tables and method): CC BY 4.0. Each price is a fact published by the provider named in its row, at the source URL given per row; we do not license those pages to you. Credit the provider as the source of each price. The texts behind the token counts keep their own licences (see the tokenizer efficiency dataset); FLORES-200 sentences (CC BY-SA 4.0) are not included anywhere in our files, only our counts.

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=data-real-cost-by-language&utm_content=page-cta) ·

**Sources**

-   [Tokenizer efficiency across languages (version 4.0, 16 tokenizers)](https://broadrouter.com/data/tokenizer-efficiency/)
-   [LLM API provider price database (providers' own pricing pages)](https://broadrouter.com/data/llm-api-provider-prices/)
-   [BroadNet Inference pricing.json](https://inference.broadnet.ai/pricing.json?utm_source=broadrouter&utm_medium=referral&utm_campaign=data-real-cost-by-language&utm_content=sources)
-   [OpenAI tiktoken model table (model.py, pinned commit)](https://github.com/openai/tiktoken/blob/212b893ba940cba53476851103d2e5c1d0020c6e/tiktoken/model.py)
