BroadRouter

Phi-4 token counter

Updated

In short: This counter runs Microsoft's official Phi-4 tokenizer.json in your browser, pinned to the same commit as our tokenizer dataset. It has 100,352 token IDs and is a 1.3 MB download. English prose takes about 129 tokens per 100 words, Modern Standard Arabic about 418.

Free, no sign-up, runs in your browser: nothing you type is sent anywhere.

Try:

… tokens

Tokens and cost of this text per model
ModelTokensChars per tokenAs inputAs output
Phi-4
Phi-4 tokenizer
Price: Phi-4 at DeepInfra, cheapest listed input price (not BroadNet)
…………
DeepSeek V4.1 Flash
DeepSeek V4.1 Flash tokenizer
…………
GLM 5.3
GLM 5.3 tokenizer
…………
Qwen3.8 27B
Qwen3.8 27B tokenizer
…………
Gemma 4 VL 26B
Gemma 4 tokenizer (Google base model)
…………
GPT (o200k)
OpenAI o200k_base
…………

Cost = tokens × the model's published price per 1M input or output tokens. GPT is shown for reference only (no BroadNet price). Phi-4 models are not sold by BroadNet: its row uses the cheapest standard input price we hold for a model with that tokenizer, read from the provider's own pricing page. Sources and check dates are in the real cost by language dataset. The chat-format option adds template tokens only for the DeepSeek, GLM, Qwen, Gemma and GPT rows; Phi-4 row counts your text alone. Measured ratios for every tokenizer are in the tokenizer efficiency dataset. Rows marked ≈ are estimates from measured characters per token until you load that tokenizer; loading downloads its tokenizer.json from Hugging Face once and keeps it in your browser cache.

Create an account ($5 starting credit)

Other counters: DeepSeek · GLM · Qwen · Gemma · GPT · Llama 3 · Llama 4 · Kimi · Mistral Small · Ministral 3 · gpt-oss · Gemma 3 · MiniMax · Nemotron 3 · Qwen3 · all models. BroadNet prices synced 2026-11-14.

Report a wrong price or number (email to info@broadnet.ai, with this page's link and your inputs filled in).

How it works

  1. Paste text into the box or pick a sample. The Phi-4 tokenizer is preselected and downloads once (1.3 MB) when you start typing.
  2. The text appears split into tokens, a colour per token; tick "Show token IDs" to see the numbers.
  3. Compare the Phi-4 count with the other tokenizers in the table; load any of them for an exact count.
  4. Copy the link or download a CSV of the counts and costs.

About the Phi-4 tokenizer

The Phi-4 tokenizer is a byte-level BPE. It has 100,352 token IDs, 96 of them added or special tokens such as chat-format markers. Text is split into words, punctuation and numbers first, then merged byte by byte. Numbers are split into groups of up to three digits, so 2026 is two tokens.

On the open texts of our tokenizer efficiency dataset it needs 1,289 tokens per 1,000 English words (rank 7 of the 16 tokenizers we measured, 1 = fewest), 2,004 per 1,000 words of Python (rank 3 of 16) and 4,177 per 1,000 words of Modern Standard Arabic (rank 16 of 16). Its best showing here is Python (rank 3 of 16); its weakest is Arabic MSA (rank 16 of 16).

1 model in our price database uses this tokenizer: Phi-4. We hold 3 standard prices for them from 3 providers. The cheapest input price is $0.07 per 1M tokens, for Phi-4 on DeepInfra (checked 2026-11-14). At that price 1,000 English words cost $0.000090 to send. Phi-4 models are not sold by BroadNet. That is less than BroadNet's cheapest chat model, Gemma 4 VL 26B, at $0.00013 for the same text.

Phi-4 tokenizer facts
TokenizerPhi-4 tokenizer, a byte-level BPE
Sourcemicrosoft/phi-4 (commit 2db69c1)
Token IDs100,352, special tokens included
Models using it1: Phi-4
Cheapest input price$0.07 per 1M tokens (Phi-4, DeepInfra, checked 2026-11-14)
English prose129 tokens per 100 words (rank 7 of 16)
Arabic (MSA)418 tokens per 100 words (rank 16 of 16)
Python code200 tokens per 100 words (rank 3 of 16)
Download1.3 MB, once; then cached in your browser

Tokens per 100 words measured on the open texts of our dataset with all 16 tokenizers (2026-11-15); rank 1 = fewest tokens on the same texts. Model mapping checked 2026-11-15. Third-party prices are standard-tier prices from each provider's own page; BroadNet prices come from its pricing.json.

How much does a Phi-4 prompt cost?

input cost  = tokens × input price per 1M  ÷ 1,000,000
output cost = tokens × output price per 1M ÷ 1,000,000

Worked example: "English Wikipedia: Water" in our dataset is 274 words. The Phi-4 tokenizer splits it into 364 tokens. Sent to Phi-4 on DeepInfra at $0.07 per 1M input tokens it costs $0.000025; the same number of tokens written as output at $0.14 per 1M costs $0.000051.

Questions

Which tokenizer does Phi-4 use?

A byte-level BPE with 100,352 token IDs, published as tokenizer.json in microsoft/phi-4. This page loads that file pinned to commit 2db69c1, the commit our dataset measured.

Which models use the Phi-4 tokenizer?

Phi-4. Each was checked against the model's own Hugging Face repo on 2026-11-15: same tokenizer file, or the same vocabulary and rules with identical counts on every dataset text. Not covered: Phi-4-reasoning (different tokenizer file) and Phi-4-reasoning-plus (different tokenizer file).

What is the cheapest Phi-4 API price per token?

Of the 3 standard prices we hold for Phi-4 models, the lowest input price is $0.07 per 1M tokens (output $0.14), for Phi-4 on DeepInfra, read from the provider's pricing page on 2026-11-14. Promotional prices are not ranked. Prices differ by model size and provider: each model's price page lists every provider we hold.

How many tokens does Arabic take with Phi-4?

Modern Standard Arabic sentences take 3.03× the tokens of their English translations with this tokenizer (rank 16 of 16 on the same texts). Hindi takes 4.78× and Chinese 1.88×. Measured on parallel sentences in our open dataset (2026-11-15). Paste your own text above, or use the Arabic token counter.

Is the Phi-4 tokenizer the same as Llama 3?

No. On the 52 texts of our dataset they gave the same count on 16; in total Phi-4 needs 55.1% more tokens, and up to 97% on a single text. See the Llama 3 counter.

Does the count include chat-template tokens?

No. For Phi-4 this page counts your text only, as our dataset does. A chat request also adds the chat template's role markers and special tokens, so the billed prompt is a little longer than your text. Read usage.prompt_tokens on a real response to see the exact figure.

Go deeper

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity