---
title: "What is a batch API for LLMs? Async jobs at half price"
description: "A batch API takes a file of many LLM requests, runs them asynchronously within a time window and returns all results, usually at a discount."
url: https://broadrouter.com/glossary/batch-api/
lang: en
updated: 2026-11-16
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Batch API

Updated 2026-11-16

A batch API is an asynchronous LLM endpoint that accepts many requests at once, usually as a file, processes them within a set window instead of immediately, and returns the results together. Providers such as OpenAI and Anthropic price batch requests at a discount to their standard APIs.

Also called: batch processing API, Message Batches API, OpenAI Batch API, batch inference, async batch requests

Many LLM jobs do not need an answer now. A batch API trades speed of reply for a lower price and higher volume.

## How it works

You prepare many requests, submit them together, and collect the results later. The two best-documented versions:

|  | OpenAI Batch API | Anthropic Message Batches API |
| --- | --- | --- |
| Input | A `.jsonl` file, one request per line | A list of Messages requests |
| Price | “50% cost discount compared to synchronous APIs” | “50 percent discount on all usage” |
| Turnaround | “Each batch completes within 24 hours (and often more quickly)” | Most within 1 hour; expires if not done within 24 hours |
| Size limit | 50,000 requests, 200 MB input file | 100,000 requests or 256 MB per batch |
| Rate limits | Separate from per-model limits | Apply to queued batch requests too |

With OpenAI you upload the file, create a batch with `completion_window: "24h"`, poll its status and download the output file. Both APIs require a unique `custom_id` per request, because results can come back in a different order from the input. Anthropic keeps results for 29 days, and its batch and [prompt caching](https://broadrouter.com/glossary/cached-input-tokens/) discounts can stack.

## Why it matters

Good fits are evaluations, classifying a large dataset, bulk summaries and back-filling. OpenAI’s guide names running evaluations and classifying large datasets as typical uses. Bad fits are anything a user is waiting for.

The savings are real but bounded. A batch halves the token price on those providers; it does not reduce how many [input and output tokens](https://broadrouter.com/glossary/input-and-output-tokens/) you send. Trimming prompts still matters.

A batch also sidesteps the [rate limit](https://broadrouter.com/glossary/rate-limit/) problem. Instead of managing 429s and backoff yourself, the provider schedules the work.

## On BroadNet Inference

BroadNet’s docs do not list a batch API, a files endpoint or a batch discount. Bulk work on BroadNet means synchronous requests run in parallel within your key’s limits: 30 requests per minute and 4 in flight on a pay-as-you-go key. Handle 429 and 5xx responses with bounded exponential backoff, as the [errors and retries guide](https://broadrouter.com/guides/errors-and-retries/) shows.

For long jobs on DeepSeek V4.1 Flash, a flat plan session runs one request at a time with no token meter, so cost is fixed by time instead of volume. Use the [flat-rate vs per-token calculator](https://broadrouter.com/tools/flat-rate-vs-per-token-calculator/) to compare the two for your job size, and the [LLM batch job planner](https://broadrouter.com/tools/llm-batch-job-planner/) for the run time under your limits and the sessions a deadline needs.

## Questions

### How much cheaper is a batch API?

OpenAI and Anthropic both document a 50% discount compared with their standard APIs. Anthropic's discount applies to input, output and special tokens.

### How long does a batch take?

OpenAI states each batch completes within 24 hours, often sooner. Anthropic says most batches finish within 1 hour and expire if processing is not done within 24 hours.

### Does BroadNet have a batch API?

BroadNet's docs do not list one. You can run bulk jobs by sending requests in parallel within your key's limits, or on a flat plan for DeepSeek V4.1 Flash.

## See also

-   [Rate limit](https://broadrouter.com/glossary/rate-limit/) (term)
-   [Concurrency](https://broadrouter.com/glossary/concurrency/) (term)
-   [Throughput](https://broadrouter.com/glossary/throughput/) (term)
-   [Pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/) (term)
-   [Input and output tokens](https://broadrouter.com/glossary/input-and-output-tokens/) (term)
-   [BroadNet Inference error codes and when to retry](https://broadrouter.com/guides/errors-and-retries/) (guide)
-   [How to handle LLM API 429 errors: backoff, jitter and limiters](https://broadrouter.com/blog/handle-llm-api-429-rate-limits/) (article)
-   [How to estimate LLM API cost before you launch a feature](https://broadrouter.com/blog/estimate-llm-api-cost-before-launch/) (article)
-   [LLM batch processing without a Batch API: a resumable runner](https://broadrouter.com/blog/llm-batch-processing-without-batch-api/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-batch-api&utm_content=page-cta) ·

**Sources**

-   [OpenAI, Batch API guide (checked 2026-11-16)](https://developers.openai.com/api/docs/guides/batch)
-   [Anthropic, batch processing with the Message Batches API (checked 2026-11-16)](https://platform.claude.com/docs/en/build-with-claude/batch-processing)
-   [BroadNet Inference API reference (checked 2026-11-16)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-batch-api&utm_content=sources)
