WER answers one question: out of every 100 words someone said, how many did the transcript get wrong?
How it works
Align the model’s transcript with a correct human reference, then count three kinds of error:
- Substitutions: a wrong word in place of the right one;
- Deletions: a word that was said but is missing;
- Insertions: a word that was not said but appears.
WER = (S + D + I) / N, where N is the number of words in the reference.
Example. The reference is “please send the invoice to our office today” (8 words). The transcript reads “please send an invoice to office today now”:
| Error | Count |
|---|---|
| “the” → “an” | 1 substitution |
| “our” missing | 1 deletion |
| “now” added | 1 insertion |
WER = 3 / 8 = 0.375, or 37.5%. Wikipedia notes WER is a word-level edit distance, and because insertions are not capped, it “can be larger than 1.0.”
Tools such as the jiwer Python package compute it. Normalize both texts first: case, punctuation, numbers written as digits or words, and for Arabic, diacritics and alef or yaa forms. Normalization choices alone can change the score noticeably.
Why it matters
- Comparing models. WER is the common metric on leaderboards and in vendor claims, so it is the first filter when choosing a speech-to-text API.
- Your audio is different. The same model can score very differently on read speech and on phone calls in dialect. For Arabic, Modern Standard Arabic and dialect audio give different numbers. Test on recordings like yours.
- Not all errors are equal. WER weights every word the same. A wrong account number counts as one error, like a missing “the”. Read transcripts, not only scores.
On BroadNet Inference
BroadNet publishes its own lab measurements for BroadNet Transcribe Arabic:
| Test set | What it is | WER (%) |
|---|---|---|
| FLEURS-ar | Clean read sentences | 3.8 |
| MASC-ar | Mixed real-world Arabic speech | 21.1 |
| BroadNet dialect set | Dialect speech | 23.4 |
The gap between 3.8 and 23.4 shows why your own test matters. BroadNet also reports the model at #2 of 37 (its September 2026 figure; the live board listed it #4 of 39 on 15 November 2026) on the Open Universal Arabic ASR Leaderboard. The $5 starting credit covers about 1,190 minutes of Arabic audio, enough for a test set. The buyer’s guide has a one-hour test plan, and the Arabic guide has the code.