---
title: "What is reranking? Cross-encoder rerankers in RAG explained"
description: "Reranking rescores the top results of a first-pass search with a more accurate model, usually a cross-encoder, before they reach the LLM. How it works."
url: https://broadrouter.com/glossary/reranking/
lang: en
updated: 2026-11-15
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Reranking

Updated 2026-11-15

Reranking is a second retrieval stage in which a more accurate but slower model, usually a cross-encoder, scores each query and candidate document pair from a first-pass search and reorders them, so only the most relevant few are passed to the language model.

Also called: reranker, rerank, re-ranking, cross-encoder reranker

First-pass search is fast but rough. Reranking spends more compute on a short list to get the order right.

## How it works

A two-stage pipeline:

1.  **Retrieve.** A fast method, such as [embeddings](https://broadrouter.com/glossary/embeddings/) in a [vector database](https://broadrouter.com/glossary/vector-database/), [BM25](https://broadrouter.com/glossary/bm25/) or [hybrid search](https://broadrouter.com/glossary/hybrid-search/), returns the top 50 to 100 candidates.
2.  **Rerank.** A reranker reads the query and each candidate together and gives each pair a relevance score. Keep the top 3 to 10.

Most rerankers are cross-encoders. The Sentence Transformers docs say a cross-encoder “Generally provides superior performance compared to a Sentence Transformer (a.k.a. bi-encoder) model,” but is “Often slower” because “it requires computation for each pair rather than each text.” The docs describe cross-encoders as “often used to re-rank the top-k results” from a bi-encoder.

```python
from sentence_transformers import CrossEncoder

model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2")
ranks = model.rank(query, passages)   # passages from your first-stage search
top = [passages[r["corpus_id"]] for r in ranks[:5]]
```

The approach goes back at least to Nogueira and Cho (2019), who fine-tuned BERT as a passage re-ranker and reported a 27% relative improvement in MRR@10 over the previous best result on MS MARCO passage retrieval.

Hosted rerank APIs exist too. Cohere’s rerank endpoint “takes in a query and a list of texts and produces an ordered array with each text assigned a relevance score,” with scores normalized to the range 0 to 1 and a `top_n` field to limit results.

## Why it matters

-   **Answer quality.** The LLM can only use what you send it. Better ordering puts the right chunk in the prompt.
-   **Token cost.** With a good reranker you can send 3 chunks instead of 10, which cuts input tokens on every question.
-   **Latency.** The reranker adds a step. Its cost grows with the number of candidates, so keep that list short.

## On BroadNet Inference

BroadNet does not offer a rerank endpoint or an embeddings endpoint. Run a cross-encoder yourself, for example with Sentence Transformers, or use a hosted rerank API, then send the top chunks to a BroadNet chat model over the OpenAI-compatible Chat Completions API.

For Arabic documents, check that your reranker was trained on Arabic or multilingual data; [Arabic RAG search](https://broadrouter.com/blog/arabic-rag-search/) covers the rest of the pipeline. If your corpus is small, a long-context model such as [DeepSeek V4.1 Flash](https://broadrouter.com/models/deepseek-v4.1-flash/) may let you skip retrieval; see [long context vs RAG](https://broadrouter.com/blog/long-context-vs-rag/).

## Questions

### Why not use the cross-encoder for the whole search?

It scores each query and document pair separately, so it is too slow to run over a whole collection. The first stage narrows millions of documents to tens or hundreds, and the reranker orders only those.

### Can an LLM be the reranker?

Yes. You can ask a chat model to score or order candidates. It is usually slower and costs more per query than a small cross-encoder, so test both on your data.

### How many candidates should I rerank?

Enough that the right answer is usually among them. Measure how often the correct chunk appears in the first-stage top 20, 50 and 100, and pick the smallest number with good recall.

## See also

-   [RAG (retrieval-augmented generation)](https://broadrouter.com/glossary/rag/) (term)
-   [Embeddings](https://broadrouter.com/glossary/embeddings/) (term)
-   [Hybrid search](https://broadrouter.com/glossary/hybrid-search/) (term)
-   [BM25](https://broadrouter.com/glossary/bm25/) (term)
-   [Chunking (RAG)](https://broadrouter.com/glossary/chunking/) (term)
-   [Vector database](https://broadrouter.com/glossary/vector-database/) (term)
-   [Use LlamaIndex with BroadNet Inference through OpenAILike](https://broadrouter.com/guides/llamaindex-openailike/) (guide)
-   [Arabic RAG: normalisation, chunking and retrieval that works](https://broadrouter.com/blog/arabic-rag-search/) (article)
-   [Long context vs RAG: when 1M tokens replaces a vector DB](https://broadrouter.com/blog/long-context-vs-rag/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-reranking&utm_content=page-cta) ·

**Sources**

-   [Nogueira and Cho, Passage Re-ranking with BERT, arXiv 1901.04085 (checked 2026-11-15)](https://arxiv.org/abs/1901.04085)
-   [Sentence Transformers docs, cross-encoder usage (checked 2026-11-15)](https://sbert.net/docs/cross_encoder/usage/usage.html)
-   [Cohere API reference, rerank (checked 2026-11-15)](https://docs.cohere.com/reference/rerank)
