---
title: "What is context rot? Why LLMs degrade with long input"
description: "Context rot is the drop in LLM accuracy as input grows longer, well before the context window is full. What Chroma measured across 18 models and how to limit it."
url: https://broadrouter.com/glossary/context-rot/
lang: en
updated: 2026-11-14
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Context rot

Updated 2026-11-14

Context rot is the gradual loss of accuracy and reliability that language models show as their input gets longer, even when the task stays the same and the input is well within the context window. The term was popularised by a 2025 Chroma technical report that tested 18 models.

Also called: context degradation, long-context degradation, context decay, context window degradation

A model that can accept a million tokens does not use the millionth token as well as the first thousand. That gap is context rot.

## How it works

Chroma’s report (Hong, Troynikov and Huber, July 2025) tested 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3, on tasks that stayed equally hard while the input grew. Its headline finding: “model performance consistently degrades with increasing input length.”

Three effects stood out:

-   **Weak matches suffer most.** When the question and the relevant passage share few words, accuracy fell faster as length grew.
-   **Distractors hurt.** “Even a single distractor reduces performance relative to the baseline”, and four made it worse. A distractor is a passage on the same topic that does not answer the question.
-   **Structure matters, oddly.** Coherent filler text hurt more than the same sentences shuffled. The authors found “structural coherence consistently hurts model performance.”

This builds on earlier work such as [lost in the middle](https://broadrouter.com/glossary/lost-in-the-middle/), which showed position effects. Context rot widens the point: length itself, not only position, costs accuracy.

## Why it matters

-   **Agents.** A coding or research [agent](https://broadrouter.com/glossary/agent/) piles up file reads, logs and old tool results. Each one is a potential distractor. Accuracy can slide long before the [context window](https://broadrouter.com/glossary/context-window/) fills.
-   **Cost and latency.** Longer prompts also cost more input tokens and take longer to process, so trimming helps on three fronts at once.
-   **Long-context vs retrieval.** Sending a whole corpus is simple, but [RAG](https://broadrouter.com/glossary/rag/) that sends only relevant chunks keeps distractors out.

## On BroadNet Inference

BroadNet’s DeepSeek V4.1 Flash and GLM 5.3 accept up to 1,048,576 tokens, and Qwen3.8 27B up to 262,144. BroadNet does not trim your context, so what you send is what the model reads. Treat the limit as a ceiling, not a target. The [1M-token context window guide](https://broadrouter.com/blog/million-token-context-window-guide/) covers when a full window pays off, and [context management for long-running agents](https://broadrouter.com/blog/long-running-agent-context-management/) shows summarising and pruning patterns. The [context window calculator](https://broadrouter.com/tools/context-window-calculator/) helps size a prompt before you send it.

## Common mistakes

-   Pasting every file “just in case.” Related but irrelevant text is exactly what Chroma’s distractor tests used.
-   Testing with a needle that matches the question word for word. Real questions rarely do.

## Questions

### Is context rot the same as lost in the middle?

They are related. Lost in the middle is about where in the prompt the key fact sits. Context rot is about how total input length, distractors and structure lower accuracy overall.

### Does a bigger context window fix context rot?

No. A larger window lets you send more, but Chroma found performance falls as input grows on every model it tested. Send what the task needs, not everything that fits.

### How do agents avoid context rot?

By keeping the working context short: summarising old turns, dropping stale tool output, and loading files on demand instead of all at once.

## See also

-   [Context window](https://broadrouter.com/glossary/context-window/) (term)
-   [Lost in the middle](https://broadrouter.com/glossary/lost-in-the-middle/) (term)
-   [RAG (retrieval-augmented generation)](https://broadrouter.com/glossary/rag/) (term)
-   [AI agent](https://broadrouter.com/glossary/agent/) (term)
-   [Cached input tokens](https://broadrouter.com/glossary/cached-input-tokens/) (term)
-   [Context engineering](https://broadrouter.com/glossary/context-engineering/) (term)
-   [Context compaction](https://broadrouter.com/glossary/context-compaction/) (term)
-   [When a 1M-token context window is worth it (and when not)](https://broadrouter.com/blog/million-token-context-window-guide/) (article)
-   [Context management for long-running agents with a 1M window](https://broadrouter.com/blog/long-running-agent-context-management/) (article)
-   [Long context vs RAG: when 1M tokens replaces a vector DB](https://broadrouter.com/blog/long-context-vs-rag/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-context-rot&utm_content=page-cta) ·

**Sources**

-   [Hong, Troynikov and Huber, Context Rot: How Increasing Input Tokens Impacts LLM Performance (Chroma technical report, 2025)](https://www.trychroma.com/research/context-rot)
-   [Liu et al., Lost in the Middle: How Language Models Use Long Contexts (arXiv)](https://arxiv.org/abs/2307.03172)
-   [BroadNet Inference API reference, model IDs and context](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-context-rot&utm_content=sources#models)
