---
title: "What is RAG? Retrieval-augmented generation explained"
description: "RAG fetches relevant passages from your own documents and adds them to the prompt so an LLM answers from them. How the pipeline works and when long context wins."
url: https://broadrouter.com/glossary/rag/
lang: en
updated: 2026-11-14
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# RAG (retrieval-augmented generation)

Updated 2026-11-14

RAG (retrieval-augmented generation) is a pattern in which an application first retrieves passages relevant to a question from an external store, such as a search or vector index, and then adds them to the prompt so a language model can answer from that text rather than from memory alone.

Also called: retrieval-augmented generation, retrieval augmented generation, RAG pipeline, retrieval-augmented LLM

RAG gives a model an open book: before it answers, your code looks up the relevant pages and puts them in the prompt.

## How it works

The term comes from Lewis et al. (2020). They paired a generator model (“parametric memory”) with “a dense vector index of Wikipedia” (“non-parametric memory”) reached through a neural retriever, and reported better results on open-domain question answering than models that relied on their weights alone.

A modern RAG pipeline with an LLM API usually looks like this:

1.  **Index.** Split documents into chunks, turn each into [embeddings](https://broadrouter.com/glossary/embeddings/) and store them in a [vector database](https://broadrouter.com/glossary/vector-database/), often next to a keyword index.
2.  **Retrieve.** Embed the user’s question and fetch the top matching chunks, say 5 to 20.
3.  **Generate.** Put those chunks in the prompt with an instruction to answer only from them and cite them. Send it to the chat model.

The model never searches by itself. Retrieval quality sets the ceiling: if the right passage is not retrieved, the model cannot use it.

## Why it matters

-   **Freshness.** The model’s weights stop at its training cutoff. RAG lets it answer from today’s price list or policy.
-   **Cost.** Sending 10 chunks costs far fewer input tokens than sending the whole corpus on every question.
-   **Traceability.** Answers can cite the passages they came from, which supports [grounding](https://broadrouter.com/glossary/grounding/) and makes [hallucinations](https://broadrouter.com/glossary/hallucination/) easier to spot.
-   **Order matters.** Models use the start and end of a long prompt better than the middle. Put the best passages first. See [lost in the middle](https://broadrouter.com/glossary/lost-in-the-middle/).

## On BroadNet Inference

BroadNet covers the generation step. Its docs do not list an embeddings endpoint, so the retriever needs a separate embedding model or a keyword search. DeepSeek V4.1 Flash and GLM 5.3 accept up to 1,048,576 tokens of context, which makes generous retrieval (or no retrieval for small corpora) practical. The [long context vs RAG](https://broadrouter.com/blog/long-context-vs-rag/) post compares both on cost and recall, and [Arabic RAG search](https://broadrouter.com/blog/arabic-rag-search/) covers normalisation and hybrid retrieval for Arabic.

## Common mistakes

-   Judging the model when the retriever is at fault. Log which chunks were retrieved for each answer.
-   Chunks that cut tables or sentences in half.
-   No instruction to say “not found”. Without it, the model fills gaps from memory.

## Questions

### Is RAG the same as fine-tuning?

No. RAG adds documents to the prompt at request time and changes nothing in the model. Fine-tuning changes the model's weights. RAG is easier to update; you re-index a document instead of retraining.

### Do I need a vector database for RAG?

Not always. Keyword search, a SQL query or a small in-memory index can all serve as the retriever. A vector database helps when you have many documents and need semantic matches.

### Does a 1M-token context window replace RAG?

Sometimes. If the whole corpus fits and is queried often with caching, you can send it all. For large or fast-changing corpora, retrieval is still cheaper and faster per question.

## See also

-   [Embeddings](https://broadrouter.com/glossary/embeddings/) (term)
-   [Vector database](https://broadrouter.com/glossary/vector-database/) (term)
-   [Context window](https://broadrouter.com/glossary/context-window/) (term)
-   [Grounding](https://broadrouter.com/glossary/grounding/) (term)
-   [Hallucination](https://broadrouter.com/glossary/hallucination/) (term)
-   [Lost in the middle](https://broadrouter.com/glossary/lost-in-the-middle/) (term)
-   [Chunking (RAG)](https://broadrouter.com/glossary/chunking/) (term)
-   [Reranking](https://broadrouter.com/glossary/reranking/) (term)
-   [Hybrid search](https://broadrouter.com/glossary/hybrid-search/) (term)
-   [BM25](https://broadrouter.com/glossary/bm25/) (term)
-   [Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/) (guide)
-   [Long context vs RAG: when 1M tokens replaces a vector DB](https://broadrouter.com/blog/long-context-vs-rag/) (article)
-   [Arabic RAG: normalisation, chunking and retrieval that works](https://broadrouter.com/blog/arabic-rag-search/) (article)
-   [When a 1M-token context window is worth it (and when not)](https://broadrouter.com/blog/million-token-context-window-guide/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-rag&utm_content=page-cta) ·

**Sources**

-   [Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv, NeurIPS 2020)](https://arxiv.org/abs/2005.11401)
-   [OpenAI, vector embeddings guide (search use case)](https://developers.openai.com/api/docs/guides/embeddings)
-   [BroadNet Inference API reference, model IDs and context](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-rag&utm_content=sources#models)
