---
title: "What is an API rate limit? RPM, TPM and HTTP 429 | BroadRouter"
description: "A rate limit caps how much a client can use an API in a time window. Common LLM limits such as RPM and TPM, what a 429 means, and how to retry with backoff."
url: https://broadrouter.com/glossary/rate-limit/
lang: en
updated: 2026-11-14
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Rate limit

Updated 2026-11-14

A rate limit is a cap an API places on how much a client can use it in a time window, such as requests per minute or tokens per minute. Requests over the limit are refused, usually with HTTP 429 Too Many Requests, until usage falls back under it.

Also called: RPM, TPM, requests per minute, tokens per minute, HTTP 429, throttling

Every shared API protects itself with limits. Knowing which ones apply tells you how fast you can go and what to do when you hit one.

## How it works

LLM APIs measure several things at once. OpenAI, for example, lists:

-   **RPM:** requests per minute
-   **RPD:** requests per day
-   **TPM:** tokens per minute
-   **TPD:** tokens per day

Whichever limit you hit first applies. Many providers also cap [concurrency](https://broadrouter.com/glossary/concurrency/), the number of requests in flight.

Over the limit, the server returns `429 Too Many Requests`. MDN describes it as the client having “sent too many requests in a given amount of time.” A `Retry-After` header may say how many seconds to wait.

A safe retry loop:

```python
import random, time

for attempt in range(5):
    r = send()
    if r.status_code != 429:
        break
    wait = float(r.headers.get("Retry-After", 2 ** attempt))
    time.sleep(wait + random.random())
```

The official OpenAI SDKs already retry 429 and 5xx errors with backoff; set `max_retries` instead of writing your own loop.

## Why it matters

-   **Throughput.** Your real ceiling is the lowest of all limits. On a key limited to 30 requests a minute, as BroadNet’s pay-as-you-go keys are, 1,000 requests take at least 1,000 ÷ 30 minutes, no matter how fast the model is. The [rate limit calculator](https://broadrouter.com/tools/llm-rate-limit-calculator/) does the math for your own limits.
-   **Reliability.** Unhandled 429s look like outages. Backoff turns them into short delays.
-   **Cost.** Some providers return 429 for spending caps too, common on [pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/) keys. Read the error body before retrying.

## On BroadNet Inference

| Key type | Limits |
| --- | --- |
| Pay-as-you-go | 30 requests a minute, 4 in flight at once |
| Hourly key | 120 requests a minute, one stream, 300,000 tokens a minute |
| Unlimited plan | One request at a time per session, no token meter |

BroadNet returns 429 when a rate, concurrency, token-rate or spending limit is reached. Back off and retry. The [errors and retries guide](https://broadrouter.com/guides/errors-and-retries/) covers every status code, and [what “unlimited” really means](https://broadrouter.com/blog/what-unlimited-llm-api-really-means/) explains which limits a flat plan still has.

## Common mistakes

-   Retrying 401 or 403 errors. Those are key or policy problems, not rate limits.
-   Running many workers on one key without a shared limiter. Each worker backs off alone while the others keep the key saturated.

## Questions

### Should I retry a 429?

Yes, after a wait. Follow the Retry-After header if present; otherwise use exponential backoff with random jitter, and cap the number of attempts. Do not retry spending-limit or billing errors that share the status code.

### Do failed requests count toward the limit?

Often, yes. OpenAI's guide says failed requests still count toward per-minute limits, so retrying instantly in a loop makes things worse.

### What is the difference between a rate limit and a concurrency limit?

A rate limit counts requests or tokens over time. A concurrency limit counts requests running at the same moment. A key can hit either one first.

## See also

-   [Concurrency](https://broadrouter.com/glossary/concurrency/) (term)
-   [API key](https://broadrouter.com/glossary/api-key/) (term)
-   [LLM router](https://broadrouter.com/glossary/llm-router/) (term)
-   [Pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/) (term)
-   [Exponential backoff](https://broadrouter.com/glossary/exponential-backoff/) (term)
-   [Idempotency key](https://broadrouter.com/glossary/idempotency-key/) (term)
-   [Batch API](https://broadrouter.com/glossary/batch-api/) (term)
-   [Spend limit](https://broadrouter.com/glossary/spend-limit/) (term)
-   [Request timeout](https://broadrouter.com/glossary/request-timeout/) (term)
-   [BroadNet Inference error codes and when to retry](https://broadrouter.com/guides/errors-and-retries/) (guide)
-   [Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/) (guide)
-   [What "unlimited" really means in an LLM API plan](https://broadrouter.com/blog/what-unlimited-llm-api-really-means/) (article)
-   [Switching to an OpenAI-compatible API: a migration checklist](https://broadrouter.com/blog/openai-compatible-api-migration-checklist/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-rate-limit&utm_content=page-cta) ·

**Sources**

-   [MDN, 429 Too Many Requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/429)
-   [OpenAI, rate limits guide](https://developers.openai.com/api/docs/guides/rate-limits)
-   [BroadNet Inference API reference, limits](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-rate-limit&utm_content=sources#limits)
