BroadRouter

Self-hosting an LLM vs an API: the real cost and break-even

By the BroadNet Inference team · 5 min read · Published · Updated

A rented GPU costs the same per hour whether busy or idle, so self-hosting beats an API only at high, steady utilization. Divide the hourly price by the tokens the server really produces per hour, add engineering and on-call time, and compare with the API price per 1M tokens. For spiky or modest traffic, the API almost always wins.

“We would save money running it ourselves” is one of the most common claims in LLM cost planning, and one of the least often checked. The math is not hard. It has one variable that people guess wrong (utilization) and one cost they leave out (people). This post gives the formula, real rental prices, and a way to run it with your own numbers.

What does GPU rental cost right now?

Hourly prices from official pricing pages, fetched 2026-11-14 (RunPod) and 2026-11-16 (Lambda). A month here is 730 hours.

GPU Provider and tier (checked 2026-11-14) Per GPU-hour Per month, always on
H100 SXM RunPod Secure Cloud USD 4.39 about USD 3,205
H100 SXM RunPod Community Cloud USD 2.69 about USD 1,964
H100 PCIe RunPod Secure Cloud USD 2.89 about USD 2,110
H200 RunPod Secure Cloud USD 5.29 about USD 3,862
A100 80GB SXM RunPod Secure Cloud USD 2.19 about USD 1,599
L40S RunPod Secure Cloud USD 1.09 about USD 796
H100 SXM Lambda on-demand, 1 to 8 GPUs (in effect from 2026-11-16; checked 2026-11-16) USD 4.59 to 4.89 about USD 3,351 to 3,570

Lambda’s page (checked 2026-11-16) says these prices take effect on 2026-11-16 at 00:00 UTC; the per-GPU rate falls as the node grows, to USD 4.59 on an 8-GPU node. RunPod’s Community Cloud rates are “from” prices that vary by host. Secure and Community Cloud are different hosting tiers, so read the tier terms if you have data-handling rules. Many models need more than one GPU: an 8-GPU H100 node at the Secure Cloud rate is about USD 25,638 a month.

How do you calculate self-hosted cost per million tokens?

A GPU produces tokens at some aggregate rate across all the requests it is batching. Your cost per token is the hourly price spread over the tokens it actually produces:

cost per 1M tokens = hourly price ÷ (aggregate tokens per second × 3,600 ÷ 1,000,000) ÷ utilization

Utilization is the share of hours the GPU is doing useful work at that rate. A worked example with assumed numbers (not a benchmark: measure your own model):

  • GPU: one H100 SXM on Community Cloud, USD 2.69 an hour.
  • Assumed aggregate output: 1,500 tokens per second with good batching.
  • Tokens per hour at full load: 1,500 × 3,600 = 5.4 million.
Utilization Effective tokens per hour Cost per 1M output tokens
100% (always saturated) 5.4M USD 0.50
50% 2.7M USD 1.00
25% (busy business hours only) 1.35M USD 1.99
10% (spiky product traffic) 0.54M USD 4.98

Now compare with an API price per 1M output tokens. The LLM API pricing table shows what dozens of hosts charge for the same open weights; on BroadNet Inference, DeepSeek V4.1 Flash costs $0.52 per 1M output tokens and Gemma 4 VL 26B costs $0.22. Find the row where your self-hosted cost drops below the API price: that is your break-even utilization. The example ignores input tokens, which also take GPU time (prefill), so real self-hosted costs are higher than the table shows.

How many tokens a month do you need to break even?

Turn it around. A GPU left running costs its monthly price regardless. The API costs tokens × price. Break-even volume is:

monthly tokens to break even = monthly GPU cost ÷ API price per 1M tokens × 1,000,000

For one Community Cloud H100 (about USD 1,964 a month) against a $0.52-per-1M output price, divide 1,964 by that price to get the millions of output tokens per month you must replace before the GPU pays for itself. Do the same with your blended input and output price if your workload is input-heavy, as agents usually are; the flat-rate vs per-token post shows how to blend them.

And that GPU must actually serve the model. If the model needs four GPUs, multiply the monthly cost by four before you divide.

What costs do people leave out?

Cost What it covers How to count it
Engineering setup Serving engine, quantization choice, autoscaling, load tests One-off hours × your rate
Ongoing ops Upgrades, driver and engine updates, model refreshes Hours per month × your rate
On-call Someone wakes up when the node dies Your on-call policy
Redundancy A second node so one failure is not an outage Doubles the GPU line
Idle headroom Capacity for peaks you must keep warm Lowers utilization
Evaluation Checking a new model version before switching Hours per release

Redundancy is the one that changes the answer most. A single GPU is a single point of failure. Two GPUs halve your utilization overnight. See evaluating a model before switching for the eval work that each upgrade brings.

When is self-hosting the right call?

  • Steady, high volume. Batch pipelines that keep GPUs saturated around the clock.
  • Data must stay inside your network. A hard requirement, not a cost argument. Check whether an API with zero data retention meets your rules first.
  • A model no API serves. Your own fine-tune or a niche open-weight model.
  • Guaranteed capacity. You need throughput an API’s rate limits will not give you.

For everything else, especially spiky traffic, early products and agents that run in bursts, an API is cheaper and has no on-call rota.

Are there flat-price options between a GPU and per-token billing?

There is a middle ground between renting a GPU and paying per token. BroadNet sells an hourly key for Gemma 4 VL 26B at $0.50 an hour (one stream, 120 requests a minute, 300,000 tokens a minute), which is $360.00 if you ran it all month. It also sells unlimited plan sessions for DeepSeek V4.1 Flash at $60 a month, each running one request at a time at about 100 tokens per second. These give a predictable bill like a rented GPU, with no setup or ops, but with fixed concurrency per key or session rather than a whole GPU’s batch capacity.

Where BroadNet Inference fits

BroadNet Inference is the API side of this comparison: per-token prices, flat plans and hourly keys for open-weight models, some partner-hosted with US inference partners, behind an OpenAI-compatible API. It fits teams whose traffic would leave a rented GPU idle much of the day, or who do not want to run inference infrastructure. It does not fit if you need a custom fine-tune served, on-premises deployment, or hundreds of parallel requests on one key. Run your numbers in the LLM API cost calculator and compare with the table above. If you are still choosing between open and closed models, open-weight vs closed model API cost compares per-token prices from each provider’s own page. The self-hosting vs API calculator does the whole break-even with your measured throughput and these GPU prices.

Questions

Is it cheaper to self-host an LLM than to use an API?

Only at high, steady volume. A rented GPU bills every hour; an API bills only for tokens you use. If your GPU would sit idle most of the day, the API is cheaper even before you count engineering time.

How much does it cost to rent an H100 per hour?

On RunPod's pricing page on 2026-11-14, an H100 SXM was USD 4.39 an hour on Secure Cloud and from USD 2.69 on Community Cloud. Lambda lists on-demand H100 SXM at USD 4.59 to 4.89 per GPU-hour, depending on node size, at prices in effect from 2026-11-16. Check the pages for current rates.

How many GPUs do I need to self-host a large model?

At least enough memory for the weights plus the KV cache for your concurrent requests. Large mixture-of-experts models usually need a multi-GPU node. Check the model card for its parameter count and recommended setup.

When does self-hosting make sense even if it costs more?

When data cannot leave your network, when you need a fine-tuned or custom model no API serves, or when you need guaranteed capacity. Those are control reasons, not cost reasons.

How do I measure my GPU's real throughput?

Run your real prompts at your real concurrency with a serving engine such as vLLM or SGLang, and record total output tokens per second across all requests. Vendor peak numbers assume ideal batching.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity