Hamza is the small mark that most often separates a correct Arabic spelling from a common one. أحمد and احمد are the same name to a reader but different strings to a computer.
How it works
The Unicode Standard says “the hamza may occur above or below other letters”, and lists alef, waw and yeh as seats. Each seated form is encoded as one precomposed character:
| Character | Code point | Unicode name |
|---|---|---|
| ء | U+0621 | ARABIC LETTER HAMZA |
| أ | U+0623 | ARABIC LETTER ALEF WITH HAMZA ABOVE |
| إ | U+0625 | ARABIC LETTER ALEF WITH HAMZA BELOW |
| آ | U+0622 | ARABIC LETTER ALEF WITH MADDA ABOVE |
| ؤ | U+0624 | ARABIC LETTER WAW WITH HAMZA ABOVE |
| ئ | U+0626 | ARABIC LETTER YEH WITH HAMZA ABOVE |
Unicode also defines decompositions: أ, ؤ and ئ are each canonically equivalent to the base letter followed by U+0654 ARABIC HAMZA ABOVE. So the same visible أ can arrive as one code point or two. Unicode normalization to NFC or NFKC turns ا + U+0654 back into the single U+0623, which is why it should run before any hamza rule.
Typed text does not always write the hamza, so the same word can appear with أ, إ, آ or bare ا. That is the variation normalization targets. CAMeL Tools’ normalize_alef_ar maps إ أ ٱ آ to ا. Lucene’s ArabicNormalizer normalizes “hamza with alef seat to a bare alef”.
Why it matters
- Search. A user types اسلام and your document has إسلام. Without the alef rule, keyword search misses it. See Arabic text normalization.
- Speech scoring. The Open Universal Arabic ASR Leaderboard computes WER after “removing all punctuation as well as the diacritics, and normalizing characters with Hamzas and Maddas”. If you score your own word error rate without the same step, a correct transcript that drops a hamza counts as wrong, and your numbers will not compare with the leaderboard’s.
- The cost. Folding hamza merges words that differ only in hamza. The verbs سأل (he asked) and سال (it flowed) both become سال. The particles إن and أن both become ان. For most search this is acceptable; for spelling correction or legal text it is not.
On BroadNet Inference
BroadNet’s API passes text through unchanged; hamza handling happens in your code. Transcripts from BroadNet Transcribe Arabic are ordinary Arabic text, so apply one normalization to both your reference and the transcript before scoring. Send language=ar, as BroadNet’s docs advise. The Arabic speech-to-text guide has the request.
For correcting hamza in user text, a chat model can help; the spelling correction post shows how to check that only intended characters changed. To see what folding does to your strings, try the Arabic text normalizer.