---
title: "What is a request timeout? Timeouts for LLM API calls"
description: "A request timeout is how long a client waits for an LLM API before giving up. Connect and read timeouts, SDK defaults, 408 and 504, and how to retry safely."
url: https://broadrouter.com/glossary/request-timeout/
lang: en
updated: 2026-11-15
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Request timeout

Updated 2026-11-15

A request timeout is the longest time an HTTP client waits for a server before it abandons the request. Clients usually split it into a connect timeout for opening the connection and a read timeout for waiting on data. LLM calls need generous read timeouts because long answers take time.

Also called: read timeout, connect timeout, client timeout, HTTP timeout

Every network call needs a point where you stop waiting. For LLM calls that point is hard to choose, because a slow answer and a dead connection look the same for a while.

## How it works

HTTP clients split waiting into phases. HTTPX, a common Python HTTP client, has four:

| Timeout | HTTPX meaning | Error |
| --- | --- | --- |
| Connect | Longest time to establish a socket connection | `ConnectTimeout` |
| Read | Longest wait for a chunk of incoming data | `ReadTimeout` |
| Write | Longest wait to send a chunk of the request body | `WriteTimeout` |
| Pool | Longest wait for a connection from the pool | `PoolTimeout` |

Plain HTTPX times out after five seconds of network inactivity by default. The OpenAI Python SDK sets its own default: “By default requests time out after 10 minutes”, raises `APITimeoutError`, and retries a timed-out request twice. It also retries connection errors, 408, 409, 429 and 5xx responses by default.

```python
from openai import OpenAI
client = OpenAI(base_url="https://inference.broadnet.ai/v1", timeout=120.0, max_retries=2)
```

Servers have timeouts too. MDN defines 504 Gateway Timeout as a gateway or proxy that “did not get a response in time from the upstream server”, and 408 Request Timeout as the server wanting “to shut down this unused connection”.

## Why it matters

-   **Reasoning models are slow to start.** Thinking happens before the first visible token. A short read timeout on a non-streamed call can cut off a healthy request.
-   **Streams detect stalls.** With [streaming](https://broadrouter.com/glossary/streaming/), each chunk resets the read timer, so a stall shows up fast even on a long answer.
-   **Retries can double cost.** A timeout does not mean the server stopped. Retrying a request that is still running may bill twice. Use [exponential backoff](https://broadrouter.com/glossary/exponential-backoff/) with a cap, and an [idempotency key](https://broadrouter.com/glossary/idempotency-key/) where an API supports one.

## On BroadNet Inference

BroadNet’s [API reference](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-request-timeout&utm_content=body) says to read a stream until `data: [DONE]`. If the connection ends before it, “treat the response as incomplete, record the X-Request-ID”, and decide whether your application can safely make a new request. For 500, 502, 503 and 504 it says to retry with bounded exponential backoff; 429 means a rate, concurrency, token-rate or spending limit was reached. Keep the X-Request-ID for support. The [errors and retries guide](https://broadrouter.com/guides/errors-and-retries/) and the [long-running agents post](https://broadrouter.com/blog/long-running-agents-timeouts-retries/) show working settings.

## Common mistakes

-   Leaving the default and wondering why a hung call blocks a worker for ten minutes.
-   Retrying without a cap, so one bad minute becomes a flood of requests.

## Questions

### What is the default timeout in the OpenAI Python SDK?

Its README says "By default requests time out after 10 minutes", and timed-out requests are retried twice by default with a short exponential backoff.

### Should I use one timeout for every LLM call?

No. A short classification call and a long reasoning answer need different limits. With streaming, a read timeout on the gap between chunks catches stalls better than one long total limit.

### What should I do when a BroadNet stream stops early?

BroadNet says to treat a stream that ends before data: \[DONE\] as incomplete, record the X-Request-ID, and decide whether your application can safely make a new request.

## See also

-   [Exponential backoff](https://broadrouter.com/glossary/exponential-backoff/) (term)
-   [Streaming](https://broadrouter.com/glossary/streaming/) (term)
-   [Latency](https://broadrouter.com/glossary/latency/) (term)
-   [Idempotency key](https://broadrouter.com/glossary/idempotency-key/) (term)
-   [Rate limit](https://broadrouter.com/glossary/rate-limit/) (term)
-   [Model fallback](https://broadrouter.com/glossary/model-fallback/) (term)
-   [BroadNet Inference error codes and when to retry](https://broadrouter.com/guides/errors-and-retries/) (guide)
-   [Stream chat completions and handle dropped connections](https://broadrouter.com/guides/streaming-chat-completions/) (guide)
-   [Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/) (guide)
-   [Long-running agents: timeouts, retries and checkpoints that hold](https://broadrouter.com/blog/long-running-agents-timeouts-retries/) (article)
-   [How to handle LLM API 429 errors: backoff, jitter and limiters](https://broadrouter.com/blog/handle-llm-api-429-rate-limits/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-request-timeout&utm_content=page-cta) ·

**Sources**

-   [OpenAI Python SDK README, timeouts and retries (checked 2026-11-15)](https://github.com/openai/openai-python)
-   [HTTPX documentation, timeouts (checked 2026-11-15)](https://www.python-httpx.org/advanced/timeouts/)
-   [MDN, 504 Gateway Timeout (checked 2026-11-15)](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/504)
-   [MDN, 408 Request Timeout (checked 2026-11-15)](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/408)
-   [BroadNet Inference API reference, streaming and errors (checked 2026-11-15)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-request-timeout&utm_content=sources)
