BroadRouter

Arabic RAG: normalisation, chunking and retrieval that works

By the BroadNet Inference team · 6 min read · Published

Arabic RAG works when you normalise text before indexing (unify alef forms, ya and alef maqsura, ta marbuta, strip diacritics and tatweel), chunk by paragraph, combine a multilingual embedding model with keyword search, and let a long-context LLM answer from the retrieved passages with citations. Apply the same normalisation to queries.

A man speaking into a microphone beside the Arabic word "حوار" (conversation), with the text "Conversations, ready to search."

To make retrieval-augmented generation (RAG) work on Arabic, normalise documents and queries with the same spelling rules, chunk by paragraph, search with both an Arabic keyword analyzer and a multilingual embedding model, and have the LLM answer only from the retrieved passages with citations.

The pattern, from the original RAG paper, is retrieve, then generate. Most Arabic RAG failures happen in the first half. A user types “الفاتوره” and the index holds “الفاتورة”; a policy says “إلغاء” and the user writes “الغاء”; the embedding model was mostly trained on English. Fix retrieval and the LLM part tends to look after itself.

What does an Arabic RAG pipeline look like?

  1. Extract text from PDFs, web pages and help-centre articles. For scanned Arabic PDFs, see Arabic OCR vs vision LLMs.
  2. Normalise a copy of the text for indexing.
  3. Chunk by structure: headings and paragraphs.
  4. Index twice: a keyword index and a vector index.
  5. Retrieve with the normalised query from both, merge and rerank.
  6. Generate an answer from the top passages, citing them.

Arabic allows several spellings of the same word, and users rarely type diacritics or hamza consistently. Normalise the indexed copy and every query with the same function:

Rule Characters Why
Unify alef أ إ آ ٱ → ا Hamza is often omitted when typing
Alef maqsura ى → ي Common in Egyptian typing and loose spelling
Ta marbuta ة → ه Users type ه at word end
Strip diacritics U+064B to U+0652, U+0670 Most text and queries are unvowelled
Remove tatweel ـ (U+0640) Decorative stretching
Digits ٠–٩ → 0–9 Match numbers in either form

Try each option on your own text with the Arabic text normalizer, which also writes a matching Python function.

import re

_DIAC = re.compile(r"[ً-ْٰ]")

def normalise(text: str) -> str:
    text = _DIAC.sub("", text).replace("ـ", "")
    text = re.sub("[أإآٱ]", "ا", text)
    text = text.replace("ى", "ي").replace("ة", "ه")
    return text.translate(str.maketrans("٠١٢٣٤٥٦٧٨٩", "0123456789"))

Code points are in the Unicode Arabic chart. Two cautions:

  • Never show the normalised text. Store the original next to it and display, cite and send the original to the LLM. Normalised Arabic looks misspelled to readers.
  • Some merges lose meaning. ة → ه merges a few word pairs, and ى → ي merges “على” (on) with “علي” (Ali). For search recall, that trade is usually worth it; test it on your own queries.

The same rules are central to measuring Arabic speech-recognition accuracy, so you can reuse one function across your stack.

How should you chunk Arabic documents?

  • Split on structure. Headings, numbered articles in contracts (المادة ١، المادة ٢) and paragraph breaks are natural boundaries.
  • A few hundred words per chunk, with a sentence or two of overlap. Arabic words carry more morphemes than English words, so a chunk covers more meaning per word than its length suggests.
  • Sentence boundaries: split on . ؟ ! and the Arabic comma ، only as a last resort. Many Arabic texts use long sentences joined with و, so paragraph-level splitting is safer.
  • Carry context. Prefix each chunk with the document title and section heading, in Arabic, so a chunk that says “تُطبَّق هذه الرسوم” (these fees apply) still says which fees.

The RAG chunking calculator turns chunk size, overlap and top-k into chunk counts, tokens per query and a monthly cost.

Why combine keyword search and embeddings?

Arabic morphology defeats naive keyword matching: كتاب, الكتاب, بالكتاب, وكتابه and كتب all relate to “book”. Use an analyzer that handles Arabic. Elasticsearch ships an Arabic language analyzer that chains Arabic stop words, digit normalisation, Arabic normalisation and an Arabic stemmer, so prefixed forms such as الكتاب and وكتاب match more often. Check what it merges on your own vocabulary before you rely on it.

Embeddings

Embedding models map text to vectors so that passages with similar meaning sit close together, even when they share no words. For Arabic, choose a multilingual model and test it:

  • Prefer models trained on many languages, such as BGE M3, which its authors describe as supporting more than 100 working languages. English-only models give poor Arabic vectors.
  • Test cross-language retrieval if users ask in English about Arabic documents, or the reverse.
  • Embed the original or the normalised text, but be consistent between documents and queries. Test both; results vary by model.

BroadNet Inference does not offer an embeddings endpoint: its documented endpoints are chat completions and audio transcriptions. Use an embedding provider you trust, or host an open multilingual embedding model in a vector database, and use BroadNet for generation. The embedding cost calculator prices indexing and queries across providers.

Merge and rerank

Run both searches, merge results with reciprocal rank fusion (score each document by the sum of 1 / (k + rank) across lists), and keep the top 10 to 30 passages. A reranker or a quick LLM relevance check can trim that further, but with long-context models you can often skip it.

How do you generate cited answers?

Give the model the passages with ids and ask it to answer only from them:

import os
from openai import OpenAI

client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])

def answer(question: str, passages: list[dict]) -> str:
    context = "\n\n".join(f"[{p['id']}] {p['title']}\n{p['text']}" for p in passages)
    system = ("أجب عن سؤال المستخدم بالعربية الفصحى المبسطة، اعتماداً فقط على المقاطع المرفقة. "
              "اذكر رقم المقطع بين قوسين مربعين بعد كل معلومة. "
              "إذا لم تجد الإجابة في المقاطع فقل ذلك بوضوح.")
    r = client.chat.completions.create(
        model="broadnet/deepseek-v4.1-flash",
        max_tokens=4000,
        messages=[{"role": "system", "content": system},
                  {"role": "user", "content": f"المقاطع:\n{context}\n\nالسؤال: {question}"}],
    )
    return r.choices[0].message.content

The Arabic system prompt says: “Answer the user’s question in simple Modern Standard Arabic, based only on the attached passages. Cite the passage number in square brackets after each fact. If you cannot find the answer in the passages, say so clearly.” English instructions work just as well; what matters is the rule to answer only from the passages and to cite.

Put the passages before the question. If the same documents appear in many requests, keep them in the same order at the start of the prompt so cached input pricing applies; see cached input tokens.

Which model?

Model Context Why
DeepSeek V4.1 Flash 1,048,576 Lowest cached-input rate of these three on BroadNet ($0.0055 per 1M tokens), room for many passages, flat plans available
GLM 5.3 1,048,576 Tools and structured output for agentic retrieval
Qwen3.8 27B 262,144 BroadNet tested it in Arabic: fluent writing, Gulf and Egyptian dialects understood

With a million-token context, you can retrieve generously, say 50 passages instead of 5, and let the model pick. That reduces the damage when retrieval ranks the right passage at number 12. For small document sets, you may not need retrieval at all; see long context vs RAG and check whether your corpus fits with the context window calculator. Prices for all models are on the pricing page.

How do you evaluate Arabic RAG?

  1. Write 50 to 100 real questions, in the mix of MSA, dialect and English your users actually type.
  2. Mark the passage that answers each one.
  3. Measure retrieval first: is the right passage in the top 10? Compare keyword only, embeddings only and hybrid, with and without normalisation.
  4. Then measure answers: correct, cited, and “not found” when the answer is missing.
  5. Include misspelled and dialect queries on purpose. Users write “وين الفاتوره” (Gulf/Levantine: where is the invoice), not “أين الفاتورة”.

Fix retrieval before you change the generation model: if the right passage never reaches the prompt, no model can cite it.

Questions

Does BroadNet Inference provide embeddings?

No. Its documented endpoints are chat completions and audio transcriptions. Use an embedding model from another provider or run an open multilingual embedding model yourself, and use BroadNet for the generation step.

Should I normalise ta marbuta to ha?

For search, usually yes, because users type both. It merges a few distinct words, but the recall gain outweighs that in most corpora. Keep the original text for display and citations.

How big should Arabic chunks be?

Start with one to three paragraphs, a few hundred words, with a small overlap. Split on headings and paragraph breaks, not mid-sentence, and keep the document title with each chunk.

Which embedding model works for Arabic?

Use a multilingual model and test it on your own Arabic queries. BGE M3, for example, is described by its authors as supporting retrieval in more than 100 working languages. English-only embedding models give poor Arabic vectors.

Can users search in English and find Arabic documents?

Yes, with a multilingual embedding model that maps both languages into one space, or by translating the query with an LLM before keyword search. Test cross-language queries explicitly.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity