LLM batch job planner
Updated
In short: A batch job takes requests ÷ the lowest of three rates: your RPM limit, TPM ÷ tokens per request, and concurrency × 60 ÷ seconds per request. 10,000 requests at BroadNet's published pay-as-you-go limit of 30 a minute take about 5.6 hours. BroadNet has no Batch API, so plan the run yourself: time, cost and sessions.
Estimates before tax. BroadNet prices synced 2026-11-14; other models use the cheapest standard price listed on a provider's own page in our database, without any batch discount. Session counts assume one request at a time per session and the speed you enter. BroadNet publishes no request or token rate for plan sessions on the pages we checked, so only latency limits them here.
Report a wrong price or number (email to info@broadnet.ai, with this page's link and your inputs filled in).
How it works
- Enter the number of items, the average input and output tokens per item (include reasoning tokens in output) and the share you expect to retry.
- Set your limits: requests in flight, requests per minute and tokens per minute, or load BroadNet's published limits for a pay-as-you-go or hourly key.
- Enter the output speed and the seconds before output starts. Measure them on a sample; the speed starts at BroadNet's stated typical figure.
- Read the run time and binding limit, the cost of the job on each BroadNet model and one model of your choice, and the flat sessions needed to finish by your deadline.
The formulas
requests = items × (1 + retry share)
seconds/request = seconds before output + output tokens ÷ output speed
rate (per minute) = min( RPM,
TPM ÷ (input + output tokens),
in flight × 60 ÷ seconds/request )
run time = requests ÷ rate
in flight needed = rate × seconds/request ÷ 60 (Little's law)
sessions needed = ⌈ requests ÷ ⌊ deadline seconds ÷ seconds/request ⌋ ⌉The tokens-per-minute line counts input and output tokens. Some providers count the max_tokens you request instead of the tokens you get back, which makes that limit bind sooner: check your provider's documentation.
Worked example
Classify 10,000 support tickets with 1,500 input and 300 output tokens each, on one BroadNet pay-as-you-go key (30 requests a minute, 4 in flight). At about 100 tokens a second plus 1 second before output, each request takes 4 seconds, so 4 in flight could reach 60 a minute, but the 30-a-minute limit binds: the run takes about 5 h 33 min. On DeepSeek V4.1 Flash at pay-as-you-go prices the job costs $4.18. On a DeepSeek session, one request at a time, one session finishes within 24 hours in about 11 h 7 min: 12 Meteor session hours at $1, $12 in total. Here pay as you go costs less and finishes sooner, because several requests run at once; sessions pay off when output is long or the work runs for weeks.
How to run a batch without a Batch API
- Measure 50 to 100 real items first: average input, output and reasoning tokens, and seconds per request.
- Run a worker pool capped at your in-flight limit and pace starts to the requests-per-minute limit.
- On a 429, back off with jitter and retry; on a 5xx, retry with bounded exponential backoff. See BroadNet errors and retries.
- Write results as they arrive, keyed by item ID, so a restart skips finished items.
- On flat sessions, give each worker its own session: one session serves one request at a time.
Questions
Does BroadNet Inference have a Batch API?
No. BroadNet's developer reference documents /v1/chat/completions, /v1/audio/transcriptions and GET /v1/models, plus account endpoints, but no batch or file-upload endpoint (checked 2026-11-15). You run a batch by sending the requests yourself, within your key's limits or on flat sessions. Batch prices and turnaround times from other providers do not apply.
How long will my batch job take?
Divide the requests (retries included) by the lowest of three rates: your requests-per-minute limit, your tokens-per-minute limit ÷ tokens per request, and concurrency × 60 ÷ seconds per request. Seconds per request is the time before output starts plus output tokens ÷ output speed. The planner shows which limit binds.
What limits does a BroadNet pay-as-you-go key have?
BroadNet publishes 30 requests per minute and 4 requests in flight per pay-as-you-go key. An hourly key for BroadNet Gemma 4 VL 26B allows one concurrent stream, 120 requests per minute and 300,000 tokens per minute. Your key details show the limits that apply to you.
How many DeepSeek sessions do I need to finish by a deadline?
A session runs one request at a time, so one session finishes (deadline in seconds ÷ seconds per request) requests. Sessions needed = requests ÷ that number, rounded up. Each extra session costs the same. The planner prices them as Full Moon ($60 a month), Orbit ($15 a week), Meteor ($1 an hour).
Is another provider's Batch API cheaper?
Often, if your results can wait. OpenAI's Batch API lists a 50% discount against its synchronous API with a 24-hour completion window, and Anthropic's Message Batches API also lists 50% off, with most batches finishing within an hour (both checked 2026-11-15). Price the same job on those models with the "Another provider's model" option, then halve it.
Why is my real run slower than the estimate?
Output length varies per item, long inputs take time to process before output starts, and 429 responses force backoff. The speed field defaults to BroadNet's stated typical figure of about 100 tokens a second for its plans, which is not a guarantee. Measure a sample of 50 to 100 items and enter your own numbers.
Go deeper
Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity
Sources
- BroadNet Inference machine-readable prices (pricing.json)
- BroadNet Inference API reference: endpoints, pay-as-you-go and hourly key limits, no batch endpoint (checked 2026-11-15)
- BroadNet Inference plans: one request at a time per session; typical speed about 100 tokens per second (checked 2026-11-15)
- OpenAI Batch API guide: 50% discount, 24-hour completion window, 50,000 requests per batch (checked 2026-11-15)
- Anthropic Message Batches API: 50% discount, most batches within an hour, 24-hour expiry (checked 2026-11-15)
- BroadNet Inference prices