---
title: "What is model fallback? Failing over to a backup LLM"
description: "Model fallback sends a request to a backup model or provider when the first one fails. How gateways like LiteLLM and OpenRouter do it, and how to build it yourself."
url: https://broadrouter.com/glossary/model-fallback/
lang: en
updated: 2026-11-16
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Model fallback

Updated 2026-11-16

Model fallback is a reliability pattern where a failed LLM request is retried on a different model or provider from an ordered list. The trigger can be an error, a rate limit, a context-length error or a content-policy refusal. It runs in your client or in a gateway in front of the APIs.

Also called: fallback model, model failover, provider fallback

Every API has bad minutes. Fallback means a bad minute at one provider does not become an outage in your app.

## How it works

You keep an ordered list of models. When a call fails, after its own retries, the next one in the list gets the same request.

Two gateways from the sources:

| Gateway | How fallback works |
| --- | --- |
| LiteLLM proxy | `fallbacks` lists model groups, tried “in-order”. Separate `context_window_fallbacks` and `content_policy_fallbacks` handle those error types. `num_retries` retries a group before falling back; `allowed_fails` and `cooldown_time` take a failing deployment out for a while. |
| OpenRouter | A `models` array. “If the first model returns an error, OpenRouter will automatically try the next model in the list.” Any error can trigger it, including context length, moderation, rate limits and downtime. If the last model fails, that error is returned. |

A minimal client-side version:

```python
from openai import APITimeoutError, InternalServerError, RateLimitError

CHAIN = [("broadnet", "broadnet/glm-5.3"), ("other", "backup-model")]

def chat_with_fallback(clients, msgs):
    for client_name, model in CHAIN:
        try:
            return clients[client_name].chat.completions.create(model=model, messages=msgs)
        except (RateLimitError, APITimeoutError, InternalServerError):
            continue
    raise RuntimeError("all models failed")
```

## Why it matters

-   **Uptime.** A second provider covers incidents and [rate limits](https://broadrouter.com/glossary/rate-limit/) at the first.
-   **Different output.** The backup model writes differently, follows tools differently and may cost more. Run your [eval](https://broadrouter.com/glossary/eval/) on every model in the chain, not only the first.
-   **Do not fall back on your own bugs.** A 400 or 422 means the request is wrong; it will fail on the next model too. Fall back on 429, timeouts and 5xx after bounded [exponential backoff](https://broadrouter.com/glossary/exponential-backoff/).

## On BroadNet Inference

BroadNet does not route requests automatically and has no fallback across models. BroadRouter, this site, is BroadNet’s information site and does not route requests either. You implement fallback yourself: in your client as above (the [LLM API failover how-to](https://broadrouter.com/blog/llm-api-failover-fallback/) adds a circuit breaker and model ID mapping), or in a gateway; see [LLM router](https://broadrouter.com/glossary/llm-router/) and the [LiteLLM proxy guide](https://broadrouter.com/guides/litellm-proxy-custom-provider/) for adding BroadNet as a provider. BroadNet’s [API reference](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-model-fallback&utm_content=body) says to retry 5xx with bounded exponential backoff and to use response details and bounded backoff on 429. The [errors and retries guide](https://broadrouter.com/guides/errors-and-retries/) lists each status.

## Common mistakes

-   Falling back on 401 or 402 errors, which hides a billing or key problem.
-   Not logging which model served each reply, so quality changes are hard to trace.

## Questions

### Should I retry or fall back first?

Retry first for short faults such as a 503. LiteLLM works this way: it falls back to another model group only after a call fails num\_retries times.

### Which model am I billed for after a fallback?

On OpenRouter, the one that served the request: it says requests "are priced using the model that was ultimately used", reported in the response model field.

### Does BroadNet route or fall back automatically?

No. BroadNet does not offer automatic routing, and BroadRouter is BroadNet's information site, not a router. Build fallback in your client or put a gateway such as LiteLLM in front.

## See also

-   [LLM router](https://broadrouter.com/glossary/llm-router/) (term)
-   [Exponential backoff](https://broadrouter.com/glossary/exponential-backoff/) (term)
-   [Rate limit](https://broadrouter.com/glossary/rate-limit/) (term)
-   [Request timeout](https://broadrouter.com/glossary/request-timeout/) (term)
-   [Inference provider](https://broadrouter.com/glossary/inference-provider/) (term)
-   [Route LiteLLM to BroadNet Inference (SDK and proxy)](https://broadrouter.com/guides/litellm-proxy-custom-provider/) (guide)
-   [BroadNet Inference error codes and when to retry](https://broadrouter.com/guides/errors-and-retries/) (guide)
-   [LLM API failover: fall back to a second provider in your own code](https://broadrouter.com/blog/llm-api-failover-fallback/) (article)
-   [OpenRouter vs a flat-rate plan for AI agents: which costs less?](https://broadrouter.com/blog/openrouter-alternative-flat-rate/) (article)
-   [How to choose an LLM API provider: a 20-point checklist](https://broadrouter.com/blog/choose-llm-api-provider-checklist/) (article)
-   [How to evaluate an LLM before switching providers](https://broadrouter.com/blog/evaluate-llm-before-switching/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-model-fallback&utm_content=page-cta) ·

**Sources**

-   [LiteLLM docs, proxy reliability: retries and fallbacks (checked 2026-11-15)](https://docs.litellm.ai/docs/proxy/reliability)
-   [OpenRouter docs, model fallbacks (checked 2026-11-15)](https://openrouter.ai/docs/guides/routing/model-fallbacks)
-   [BroadNet Inference API reference, errors and retries (checked 2026-11-15)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-model-fallback&utm_content=sources)
