---
title: "Long-running AI agents - timeouts, retries, checkpoints"
description: "Keep an LLM agent running for hours: timeouts per call, streaming to spot stalls, safe retries, idempotent tool side effects and a checkpoint after every step."
url: https://broadrouter.com/blog/long-running-agents-timeouts-retries/
lang: en
updated: 2026-11-14
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Long-running agents: timeouts, retries and checkpoints that hold

By the BroadNet Inference team · 4 min read · Published 2026-11-14 · Updated 2026-11-16

Treat each model call as disposable and the agent's state as durable. Set a timeout per call, stream to detect stalls, retry 429, 5xx and dropped streams with bounded backoff, and give every side-effecting tool an idempotency key. Save the message list after each step so a crash resumes from the last checkpoint, not from the start.

## Key takeaways

-   The OpenAI Python SDK times out after 10 minutes and retries twice by default. Set both explicitly for agents.
-   A stream that ends before data \[DONE\] is an incomplete answer. Retrying it is a new inference, billed again on pay-as-you-go.
-   Model calls are safe to retry. Tool side effects are not, unless they carry an idempotency key.
-   Checkpoint after every completed step: the message list, the step number and the IDs of finished side effects.
-   On a plan session, retries cost nothing extra but occupy the one request slot while they run.

**On this page**

1.  What breaks when an agent runs for hours?
2.  1\. How long should a call be allowed to take?
3.  2\. Why stream in a long-running agent?
4.  3\. What should you retry?
5.  4\. How do you make tool calls idempotent?
6.  5\. How do you checkpoint an agent?
7.  A checklist before an overnight run
8.  Where BroadNet Inference fits

An agent that runs for five minutes can ignore failure. One that runs for five hours will see a timeout, a 429, a dropped stream and probably a restart. Most guides on long-running agents sell a workflow engine. You may not need one. This post covers the five habits that keep a plain agent loop alive for hours, with code for the OpenAI Python SDK.

## What breaks when an agent runs for hours?

| Failure | How it looks | Habit that handles it |
| --- | --- | --- |
| Hung or slow call | No reply for minutes | Per-call timeout, streaming |
| Rate limit or overload | 429, 5xx | Bounded retry with backoff |
| Dropped stream | Connection ends before `[DONE]` | Treat as incomplete, retry the call |
| Duplicate side effect | Tool runs twice after a retry | Idempotency keys |
| Process crash or deploy | Agent restarts at step 1 | Checkpoint per step |

## 1\. How long should a call be allowed to take?

The OpenAI Python SDK’s README says requests time out after 10 minutes by default and certain errors are retried twice. For an agent, set both yourself, so the behavior doesn’t change under you:

```python
client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"],
                timeout=300, max_retries=4)
```

Pick the timeout from data. Log the duration of every call for a day; set the timeout well above the slowest successful call. Large prompts take longer to start, and [reasoning models](https://broadrouter.com/glossary/reasoning-tokens/) think before writing. At about 100 tokens per second, 8,000 output tokens take over a minute on their own. Add a separate wall-clock budget per **task** so a loop that keeps going can’t run all night.

## 2\. Why stream in a long-running agent?

[Streaming](https://broadrouter.com/glossary/streaming/) gives you a signal while the model works. Without it, a slow call and a dead connection look the same until the timeout fires. With it, you can see chunks arrive and also get token usage in the last chunk.

BroadNet’s streaming rules: read until the literal `data: [DONE]` line. If the connection closes before it, the answer is incomplete. Record the `X-Request-ID`, then decide whether to make a new request. [Full streaming guide](https://broadrouter.com/guides/streaming-chat-completions/).

```python
def call_streaming(messages, tools=None):
    parts, done = [], False
    stream = client.chat.completions.create(
        model="broadnet/glm-5.3", messages=messages, tools=tools, stream=True,
        stream_options={"include_usage": True}, max_tokens=8000)
    for chunk in stream:
        if chunk.choices and chunk.choices[0].finish_reason:
            done = True
        if chunk.choices and chunk.choices[0].delta.content:
            parts.append(chunk.choices[0].delta.content)
    if not done:
        raise IncompleteStream()   # retry the whole call
    return "".join(parts)
```

This sketch collects text only. If you stream tool calls, accumulate `delta.tool_calls` by index as well.

## 3\. What should you retry?

The SDK already retries connection errors, 408, 409, 429 and 5xx. Layer your own retry on top only for what it can’t see, such as an incomplete stream:

1.  **Retry:** connection errors, 429, 5xx, incomplete streams. Use [exponential backoff](https://broadrouter.com/glossary/exponential-backoff/) with jitter and a cap, for example 4 attempts.
2.  **Fix, don’t retry:** 400, 401, 402, 403, 413, 422. These fail the same way every time. See [error codes and when to retry](https://broadrouter.com/guides/errors-and-retries/).
3.  **Budget retries per task**, not only per call. Ten steps each retrying four times is forty extra calls.
4.  **Fail over once retries are spent.** If a run must survive a provider outage, move connection errors, 5xx and rate-limit 429s to a second provider. [LLM API failover](https://broadrouter.com/blog/llm-api-failover-fallback/) shows the client code and a circuit breaker.

On pay-as-you-go, every retry is a new inference, billed again. A pay-as-you-go key also allows 30 requests a minute and 4 in flight, so retry storms hit the [rate limit](https://broadrouter.com/glossary/rate-limit/) quickly; the [rate limit calculator](https://broadrouter.com/tools/llm-rate-limit-calculator/) shows how much headroom your agent has. The [429 handling post](https://broadrouter.com/blog/handle-llm-api-429-rate-limits/) covers backoff in detail. On an [unlimited plan](https://broadrouter.com/guides/unlimited-llm-api-sessions/), retries cost nothing extra, but they occupy the session’s single request slot while they run.

## 4\. How do you make tool calls idempotent?

A model call has no side effects. A tool call can. If the agent crashes after `send_invoice` ran but before the step was saved, the restarted agent will send it again.

Give each side-effecting call an [idempotency key](https://broadrouter.com/glossary/idempotency-key/) derived from the task and step, check it before acting, and skip the call if it’s already recorded:

```python
def run_side_effect(task_id, step, call, store):
    key = f"{task_id}:{step}:{call.id}"
    if store.done(key):
        return store.result(key)       # already ran: return the saved result
    result = execute(call)             # pass key to the API if it supports one
    store.save(key, result)
    return result
```

Many payment and messaging APIs accept an idempotency key header. Pass yours through when they do.

## 5\. How do you checkpoint an agent?

Save after every completed step: the message list, the step number and the side-effect keys. On start, load the latest checkpoint if one exists.

```python
def save_checkpoint(path, step, messages):
    tmp = path + ".tmp"
    with open(tmp, "w") as f:
        json.dump({"step": step, "messages": messages}, f)
    os.replace(tmp, path)   # atomic on the same filesystem
```

Writing to a temp file and renaming means a crash mid-write never leaves a half checkpoint. For very long sessions, also checkpoint a **summary**: replace old turns with a short summary so a resumed agent doesn’t reload a huge history. That keeps you inside the [context window](https://broadrouter.com/glossary/context-window/), even at 1,048,576 tokens on [DeepSeek V4.1 Flash](https://broadrouter.com/models/deepseek-v4.1-flash/).

## A checklist before an overnight run

1.  Timeout and `max_retries` set explicitly on the client.
2.  Streaming on, with `include_usage`, and incomplete streams retried.
3.  A retry budget per task and a wall-clock limit.
4.  Idempotency keys on every tool that writes, sends or pays.
5.  A checkpoint after every step, written atomically.
6.  Logs with step number, duration, tokens and `X-Request-ID`.

If you run several agents side by side, read [running parallel coding agents on a flat rate](https://broadrouter.com/blog/run-parallel-coding-agents-flat-rate/) and [what multi-agent setups cost](https://broadrouter.com/blog/multi-agent-cost/).

## Where BroadNet Inference fits

BroadNet Inference uses standard chat-completions streaming, returns an `X-Request-ID` header you can log, and documents which errors to retry, so the patterns above work as written. For agents that run all day, the unlimited plans remove the token meter on DeepSeek V4.1 Flash for $60 a month per session, which makes retries free. Keep in mind that a Meteor hour ends one hour after purchase and an Orbit week after seven days, so a long job must finish inside the period or move to a renewing Full Moon session. Plans are at [inference.broadnet.ai/plans](https://inference.broadnet.ai/plans?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-long-running-agents-timeouts-retries&utm_content=body).

Free tools for this

-   [Agent run cost estimator](https://broadrouter.com/tools/agent-run-cost-estimator/) Context growth per step, total tokens and cost of one agent run, with a DeepSeek session comparison.
-   [LLM batch job planner](https://broadrouter.com/tools/llm-batch-job-planner/) Run time, binding limit, cost per model and the flat sessions needed to finish a big job by a deadline.
-   [Flat-rate vs per-token LLM cost calculator](https://broadrouter.com/tools/flat-rate-vs-per-token-calculator/) Break-even between an unlimited plan and paying per token, per agent.
-   [LLM rate limit and concurrency calculator](https://broadrouter.com/tools/llm-rate-limit-calculator/) Which limit binds first: RPM, TPM or concurrency, with Little's law.

[All free tools](https://broadrouter.com/tools/)

## Terms in this article

[Token](https://broadrouter.com/glossary/token/)

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

[Streaming](https://broadrouter.com/glossary/streaming/)

Streaming is a way of returning a language model response in small chunks while it is still being generated, instead of as one reply at the end. OpenAI-compatible APIs stream over server-sent events when the request sets stream to true.

[Idempotency key](https://broadrouter.com/glossary/idempotency-key/)

An idempotency key is a unique value, usually a random ID such as a UUID, that a client attaches to a request that changes state, such as a payment or a purchase. If the same request is sent again with the same key, the server returns the original result instead of performing the action twice.

[Pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/)

Pay-as-you-go is a pricing model in which you pay only for what you use, metered per unit such as tokens, requests or audio minutes, with no subscription or minimum commitment. Most LLM APIs bill this way, from a prepaid balance or a periodic invoice.

[Rate limit](https://broadrouter.com/glossary/rate-limit/)

A rate limit is a cap an API places on how much a client can use it in a time window, such as requests per minute or tokens per minute. Requests over the limit are refused, usually with HTTP 429 Too Many Requests, until usage falls back under it.

[Exponential backoff](https://broadrouter.com/glossary/exponential-backoff/)

Exponential backoff is a retry strategy in which a client waits longer after each failed attempt, typically doubling the delay up to a cap, before trying again. Adding random jitter to each wait spreads out retries from many clients so they do not hit the server at the same moment.

[AI agent](https://broadrouter.com/glossary/agent/)

An AI agent is a system in which a language model directs its own steps toward a goal: it decides which tools to call, reads the results and repeats until it produces a final answer or hits a limit. A fixed script that calls a model at set points is usually called a workflow instead.

[AI coding agent](https://broadrouter.com/glossary/coding-agent/)

An AI coding agent is a software tool in which a language model works on a codebase in a loop: it reads files, edits them, runs commands such as tests, reads the output and decides the next step until the task is done or a limit is reached.

## Questions

### What timeout should I set for LLM API calls in an agent?

Long enough for the slowest legitimate call, short enough to catch a hung one. The OpenAI Python SDK defaults to 10 minutes. Many agents use 2 to 5 minutes per call with streaming, and a separate wall-clock limit per task.

### Which LLM API errors should an agent retry?

Retry connection errors, 429 and 5xx with bounded exponential backoff and jitter. Fix 400, 401, 402, 403, 413 and 422 instead of retrying. On BroadNet, also log the X-Request-ID header for each failure.

### Is it safe to retry a chat completion?

The model call itself has no side effects, so yes, but it is a new inference and is billed again on pay-as-you-go. The danger is in tools. A retried step can send an email twice unless the tool uses an idempotency key.

### How do I resume an agent after a crash?

Save the message list and step number after every completed step, write side-effect IDs before you act, and reload them on start. The agent resumes from the last saved step.

### Do long-running agents need an unlimited plan?

No, but retries and long contexts add up on pay-as-you-go. On BroadNet's unlimited plans, DeepSeek V4.1 Flash has no token meter, so retries cost nothing extra, though each session still runs one request at a time.

## Keep reading

-   [Tool calling with open models: schemas, parallel calls, retries](https://broadrouter.com/blog/tool-calling-open-models/)
-   [What multi-agent setups really cost: per token vs flat sessions](https://broadrouter.com/blog/multi-agent-cost/)
-   [How to run parallel coding agents on a flat-rate LLM API](https://broadrouter.com/blog/run-parallel-coding-agents-flat-rate/)
-   [What "unlimited" really means in an LLM API plan](https://broadrouter.com/blog/what-unlimited-llm-api-really-means/)
-   [LLM API failover: fall back to a second provider in your own code](https://broadrouter.com/blog/llm-api-failover-fallback/)
-   [Claude Code with open models: proxies, native endpoints and limits](https://broadrouter.com/blog/claude-code-with-openai-compatible-models/)
-   [OpenClaw and Hermes Agent on a cheap model: setup and token math](https://broadrouter.com/blog/openclaw-hermes-agent-cheap-model/)
-   [Guide: BroadNet Inference error codes and when to retry](https://broadrouter.com/guides/errors-and-retries/)
-   [Guide: Stream chat completions and handle dropped connections](https://broadrouter.com/guides/streaming-chat-completions/)
-   [Guide: How the unlimited LLM API plans work for agents](https://broadrouter.com/guides/unlimited-llm-api-sessions/)
-   [Guide: Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-long-running-agents-timeouts-retries&utm_content=page-cta) ·

**Sources**

-   [OpenAI Python SDK README, retries and timeouts (checked 2026-11-14)](https://github.com/openai/openai-python)
-   [BroadNet Inference API reference, errors and retries](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-long-running-agents-timeouts-retries&utm_content=sources#errors)
-   [BroadNet Inference API reference, streaming and limits](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-long-running-agents-timeouts-retries&utm_content=sources#requests)
-   [BroadNet Inference unlimited plans](https://inference.broadnet.ai/plans?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-long-running-agents-timeouts-retries&utm_content=sources)
