BroadRouter

Arabic–English translation with LLMs: prompts, glossaries, evaluation

By the BroadNet Inference team · 5 min read · Published

To translate Arabic and English with an LLM, give the model the audience, register and a glossary of fixed terms, ask it to keep numbers and names unchanged, and translate in paragraph-sized chunks. Evaluate with chrF or COMET plus native review. In BroadNet's tests, Qwen3.8 27B translated accurately between Arabic and English.

To translate between Arabic and English with an LLM, tell the model the direction, audience and register, give it a glossary of fixed terms, keep numbers and names unchanged, and send paragraph-sized chunks. Then check numbers and negations in code and score with chrF, COMET and a native reviewer.

That control matters in Arabic, where a literal translation can be correct and still read as foreign.

Why is Arabic–English translation tricky for LLMs?

  • Register. Formal Arabic (Modern Standard Arabic, فصحى) and everyday speech differ widely. A marketing line that is casual in English can become stiff or comic in literal MSA.
  • Word order and length. Arabic sentences often start with the verb, and Arabic text is often a different length from the English. UI strings that fit in English may wrap or overflow.
  • Gender and number. Arabic marks gender on verbs, adjectives and pronouns, and has a dual. “You” in English must become أنتَ, أنتِ, أنتما, أنتم or أنتنّ. If you don’t know the reader, tell the model which default to use.
  • Names and transliteration. “محمد” has many English spellings (Mohammed, Muhammad, Mohamed). Pick one per person and keep it.
  • Numbers and dates. Eastern Arabic numerals, Hijri dates and right-to-left formatting all need explicit rules.

What prompt gives good Arabic translations?

Translation quality depends far more on the instructions than on clever wording. Tell the model the direction, audience, register, glossary and output format:

import os
from openai import OpenAI

client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])

GLOSSARY = """Invoice = فاتورة
VAT = ضريبة القيمة المضافة
Refund = استرداد المبلغ
Terms and Conditions = الشروط والأحكام
BroadPay = BroadPay (do not translate)"""

SYSTEM = f"""You translate customer-support articles from English to Arabic.
Audience: retail customers in Saudi Arabia and the UAE.
Register: clear, polite Modern Standard Arabic; short sentences; address the reader
in the masculine singular as a neutral default.
Rules:
- Use the glossary exactly. Never translate product names marked "do not translate".
- Keep every number, date, URL, email address and code snippet unchanged.
- Keep Markdown structure (headings, lists, links) unchanged.
- Output only the translation.
Glossary:
{GLOSSARY}"""

def translate(text: str) -> str:
    r = client.chat.completions.create(
        model="broadnet/qwen3.8-27b",
        max_tokens=4000,  # the model reasons before answering
        messages=[{"role": "system", "content": SYSTEM},
                  {"role": "user", "content": text}],
    )
    return r.choices[0].message.content

For Arabic → English, flip the direction and add what the English reader needs: “American English, plain style, keep Arabic personal names in the spelling given in the glossary”. If the source may be dialect, say so: “The source may be Gulf or Egyptian Arabic; translate the meaning, not word for word.”

Why does a glossary matter so much?

Without a glossary, a model may translate “refund” as استرجاع in one paragraph and استرداد in the next. Both are fine Arabic; together they look careless and break search. A glossary fixes that:

  1. Start from your existing translations. Extract terms from the help centre, contracts and UI strings you already have in both languages.
  2. Keep it short. Fifty to two hundred terms covers most products. Put only terms that must not vary.
  3. Include do-not-translate items. Brand names, product names, plan names and code identifiers.
  4. Place it in the system prompt, before the text. That also keeps it in the cacheable prefix, which lowers the input cost on repeated calls; see cached input tokens.
  5. Check it in code. After translation, verify each glossary term that appeared in the source appears in its fixed form in the target.

How should you split long documents?

Translate in paragraph or section chunks, not sentence by sentence. Sentence-level calls lose context (who “he” is, which product “it” refers to). Whole-document calls work on long-context models, but a single mistake is harder to find and retry. A section of a few hundred words is a good balance. Pass the previous section’s translation as context if consistency matters.

Qwen3.8 27B has a 262,144-token context; DeepSeek V4.1 Flash and GLM 5.3 take up to 1,048,576 tokens, which is enough to include a long reference document alongside the text you translate.

How do you catch the dangerous errors?

Fluent output hides mistakes. Check these automatically:

  • Numbers: extract every number from source and target (after converting Eastern Arabic digits) and compare the sets.
  • Negation: a dropped “not” (لا, لم, لن, ليس, ما) reverses the meaning. Flag sentences where the source has a negation and the target does not.
  • Names and glossary terms: verify they survived unchanged.
  • Length ratio: a target far shorter than expected usually means something was skipped.
import re
EASTERN = str.maketrans("٠١٢٣٤٥٦٧٨٩", "0123456789")

def numbers(s: str) -> list[str]:
    return sorted(re.findall(r"\d+(?:[.,]\d+)?", s.translate(EASTERN)))

def check(src: str, tgt: str) -> list[str]:
    issues = []
    if numbers(src) != numbers(tgt):
        issues.append(f"numbers differ: {numbers(src)} vs {numbers(tgt)}")
    return issues

The decimal separator can differ between locales (Arabic text sometimes uses ٫), so normalise those too if your documents use them.

How do you evaluate translation quality?

Automatic metrics tell you whether a change to prompt or model made things better or worse on average:

Metric What it measures Notes for Arabic
BLEU Word n-gram overlap with a reference Harsh on Arabic, where clitics and spelling variants change word forms
chrF Character n-gram overlap More forgiving of Arabic morphology; a better default
COMET A neural model’s quality judgement Correlates better with human ratings; needs a GPU or a hosted scorer

For a public benchmark, FLORES-200, used by the No Language Left Behind project, includes Modern Standard Arabic and Egyptian, Najdi, North and South Levantine, Mesopotamian, Moroccan, Tunisian and Ta’izzi-Adeni Arabic alongside English, according to its language list. But your own content is the real test:

  1. Take 100 to 200 representative segments with existing professional translations.
  2. Translate them with each model and prompt version.
  3. Score chrF (and COMET if you can), after normalising Arabic spelling the same way for all systems.
  4. Have a native speaker rate 30 segments per system for accuracy and naturalness, blind to which system produced them.
  5. Track the dangerous-error checks above as a separate count.

Which model should you use, and what will it cost?

BroadNet reports that Qwen3.8 27B translated accurately between Arabic and English, wrote fluent Arabic and understood Gulf and Egyptian dialects in its tests. That makes it the natural first candidate on BroadNet Inference. DeepSeek V4.1 Flash is worth testing too: its output tokens are cheaper, and it is the model on the unlimited flat plans. The DeepSeek V4.1 Flash vs Qwen3.8 27B comparison sets the two side by side.

Per 1M tokens:

Model Input Output
Qwen3.8 27B $0.175 $2.17
DeepSeek V4.1 Flash $0.175 $0.52
GLM 5.3 $0.78 $3.46

Translation output is about as long as the input, and all three models reason before answering, so output tokens usually dominate the bill. Arabic also tends to need more tokens per word than English; see tokens per word in Arabic and English. Paste a sample into the free Arabic token counter, confirm with usage from a real request, then project monthly spend with the LLM API cost calculator. For a large one-off localisation project on DeepSeek V4.1 Flash, the $15 weekly plan removes the token meter entirely.

Can you send contracts and tickets to the API?

Contracts, medical letters and support tickets often contain personal data. BroadNet Inference does not retain prompts or outputs after the request; see zero data retention. Check that your translation memory and logs follow the same rules.

Questions

Is an LLM better than a dedicated translation API for Arabic?

For short, generic text the difference can be small. LLMs win when you need control over tone, glossary terms, formatting or dialect input, because you can say exactly what you want in the prompt.

Can the model translate dialect Arabic into English?

Often yes, because translation depends on understanding, and models understand dialects better than they write them. BroadNet reports Qwen3.8 27B understood Gulf and Egyptian dialects in its tests. Check a sample with native speakers.

Should legal documents be machine translated?

Use an LLM for a first draft and for comprehension, but have a qualified translator review anything with legal effect. Many jurisdictions also require certified human translation for official filings.

Why is my Arabic translation longer in tokens than the English?

Tokenizers often split Arabic into more tokens per word than English. Paste a sample into a token counter, then confirm with usage.prompt_tokens and usage.completion_tokens from a real request.

Which metric should I use to evaluate Arabic machine translation?

Use chrF as the default automatic score, because character n-grams cope with Arabic clitics and spelling variants better than BLEU. Add COMET if you can run it, and have native speakers blind-rate 30 or more segments per system.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity