---
title: "Self-hosting LLM vs API cost - GPU rental break-even math"
description: "GPU rental prices from official pages, the formula for cost per 1M tokens on your own GPU, the utilization you need to beat an API, and the ops costs people forget."
url: https://broadrouter.com/blog/self-hosting-llm-vs-api-cost/
lang: en
updated: 2026-11-14
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Self-hosting an LLM vs an API: the real cost and break-even

By the BroadNet Inference team · 5 min read · Published 2026-11-14 · Updated 2026-11-16

A rented GPU costs the same per hour whether busy or idle, so self-hosting beats an API only at high, steady utilization. Divide the hourly price by the tokens the server really produces per hour, add engineering and on-call time, and compare with the API price per 1M tokens. For spiky or modest traffic, the API almost always wins.

## Key takeaways

-   On RunPod's pricing page, one H100 SXM costs USD 4.39 an hour on Secure Cloud and USD 2.69 on Community Cloud: about USD 3,205 and USD 1,964 a month if left running.
-   Self-hosted cost per 1M tokens = hourly price ÷ (tokens per second × 3,600 ÷ 1,000,000) ÷ utilization.
-   Utilization decides the answer. A GPU busy 25% of the time costs four times as much per token as one busy all the time.
-   Large open-weight models need several GPUs just to hold their weights; multiply the rental price accordingly.
-   Engineering, monitoring, upgrades and on-call are real costs; count them as hours per month at your own rate.

**On this page**

1.  What does GPU rental cost right now?
2.  How do you calculate self-hosted cost per million tokens?
3.  How many tokens a month do you need to break even?
4.  What costs do people leave out?
5.  When is self-hosting the right call?
6.  Are there flat-price options between a GPU and per-token billing?
7.  Where BroadNet Inference fits

“We would save money running it ourselves” is one of the most common claims in LLM cost planning, and one of the least often checked. The math is not hard. It has one variable that people guess wrong (utilization) and one cost they leave out (people). This post gives the formula, real rental prices, and a way to run it with your own numbers.

## What does GPU rental cost right now?

Hourly prices from official pricing pages, fetched 2026-11-14 (RunPod) and 2026-11-16 (Lambda). A month here is 730 hours.

| GPU | Provider and tier (checked 2026-11-14) | Per GPU-hour | Per month, always on |
| --- | --- | --- | --- |
| H100 SXM | RunPod Secure Cloud | USD 4.39 | about USD 3,205 |
| H100 SXM | RunPod Community Cloud | USD 2.69 | about USD 1,964 |
| H100 PCIe | RunPod Secure Cloud | USD 2.89 | about USD 2,110 |
| H200 | RunPod Secure Cloud | USD 5.29 | about USD 3,862 |
| A100 80GB SXM | RunPod Secure Cloud | USD 2.19 | about USD 1,599 |
| L40S | RunPod Secure Cloud | USD 1.09 | about USD 796 |
| H100 SXM | Lambda on-demand, 1 to 8 GPUs (in effect from 2026-11-16; checked 2026-11-16) | USD 4.59 to 4.89 | about USD 3,351 to 3,570 |

Lambda’s page (checked 2026-11-16) says these prices take effect on 2026-11-16 at 00:00 UTC; the per-GPU rate falls as the node grows, to USD 4.59 on an 8-GPU node. RunPod’s Community Cloud rates are “from” prices that vary by host. Secure and Community Cloud are different hosting tiers, so read the tier terms if you have data-handling rules. Many models need more than one GPU: an 8-GPU H100 node at the Secure Cloud rate is about USD 25,638 a month.

## How do you calculate self-hosted cost per million tokens?

A GPU produces tokens at some aggregate rate across all the requests it is batching. Your cost per token is the hourly price spread over the tokens it actually produces:

**cost per 1M tokens = hourly price ÷ (aggregate tokens per second × 3,600 ÷ 1,000,000) ÷ utilization**

Utilization is the share of hours the GPU is doing useful work at that rate. A worked example with **assumed** numbers (not a benchmark: measure your own model):

-   GPU: one H100 SXM on Community Cloud, USD 2.69 an hour.
-   Assumed aggregate output: 1,500 tokens per second with good batching.
-   Tokens per hour at full load: 1,500 × 3,600 = 5.4 million.

| Utilization | Effective tokens per hour | Cost per 1M output tokens |
| --- | --- | --- |
| 100% (always saturated) | 5.4M | USD 0.50 |
| 50% | 2.7M | USD 1.00 |
| 25% (busy business hours only) | 1.35M | USD 1.99 |
| 10% (spiky product traffic) | 0.54M | USD 4.98 |

Now compare with an API price per 1M output tokens. The [LLM API pricing table](https://broadrouter.com/llm-api-pricing/) shows what dozens of hosts charge for the same open weights; on BroadNet Inference, [DeepSeek V4.1 Flash](https://broadrouter.com/models/deepseek-v4.1-flash/) costs $0.52 per 1M output tokens and [Gemma 4 VL 26B](https://broadrouter.com/models/gemma-4-vl-26b/) costs $0.22. Find the row where your self-hosted cost drops below the API price: that is your break-even utilization. The example ignores input tokens, which also take GPU time (prefill), so real self-hosted costs are higher than the table shows.

## How many tokens a month do you need to break even?

Turn it around. A GPU left running costs its monthly price regardless. The API costs tokens × price. Break-even volume is:

**monthly tokens to break even = monthly GPU cost ÷ API price per 1M tokens × 1,000,000**

For one Community Cloud H100 (about USD 1,964 a month) against a $0.52-per-1M output price, divide 1,964 by that price to get the millions of output tokens per month you must replace before the GPU pays for itself. Do the same with your blended input and output price if your workload is input-heavy, as agents usually are; the [flat-rate vs per-token post](https://broadrouter.com/blog/flat-rate-vs-pay-per-token-llm-api/) shows how to blend them.

And that GPU must actually serve the model. If the model needs four GPUs, multiply the monthly cost by four before you divide.

## What costs do people leave out?

| Cost | What it covers | How to count it |
| --- | --- | --- |
| Engineering setup | Serving engine, quantization choice, autoscaling, load tests | One-off hours × your rate |
| Ongoing ops | Upgrades, driver and engine updates, model refreshes | Hours per month × your rate |
| On-call | Someone wakes up when the node dies | Your on-call policy |
| Redundancy | A second node so one failure is not an outage | Doubles the GPU line |
| Idle headroom | Capacity for peaks you must keep warm | Lowers utilization |
| Evaluation | Checking a new model version before switching | Hours per release |

Redundancy is the one that changes the answer most. A single GPU is a single point of failure. Two GPUs halve your utilization overnight. See [evaluating a model before switching](https://broadrouter.com/blog/evaluate-llm-before-switching/) for the eval work that each upgrade brings.

## When is self-hosting the right call?

-   **Steady, high volume.** Batch pipelines that keep GPUs saturated around the clock.
-   **Data must stay inside your network.** A hard requirement, not a cost argument. Check whether an API with [zero data retention](https://broadrouter.com/zero-data-retention/) meets your rules first.
-   **A model no API serves.** Your own fine-tune or a niche [open-weight model](https://broadrouter.com/glossary/open-weight-model/).
-   **Guaranteed capacity.** You need throughput an API’s rate limits will not give you.

For everything else, especially spiky traffic, early products and agents that run in bursts, an API is cheaper and has no on-call rota.

## Are there flat-price options between a GPU and per-token billing?

There is a middle ground between renting a GPU and paying per token. BroadNet sells an hourly key for Gemma 4 VL 26B at $0.50 an hour (one stream, 120 requests a minute, 300,000 tokens a minute), which is $360.00 if you ran it all month. It also sells [unlimited plan sessions](https://broadrouter.com/guides/unlimited-llm-api-sessions/) for DeepSeek V4.1 Flash at $60 a month, each running one request at a time at about 100 tokens per second. These give a predictable bill like a rented GPU, with no setup or ops, but with fixed concurrency per key or session rather than a whole GPU’s batch capacity.

## Where BroadNet Inference fits

BroadNet Inference is the API side of this comparison: per-token prices, flat plans and hourly keys for open-weight models, some partner-hosted with US inference partners, behind an OpenAI-compatible API. It fits teams whose traffic would leave a rented GPU idle much of the day, or who do not want to run inference infrastructure. It does not fit if you need a custom fine-tune served, on-premises deployment, or hundreds of parallel requests on one key. Run your numbers in the [LLM API cost calculator](https://broadrouter.com/tools/llm-api-cost-calculator/) and compare with the table above. If you are still choosing between open and closed models, [open-weight vs closed model API cost](https://broadrouter.com/blog/open-weight-vs-closed-model-api-cost/) compares per-token prices from each provider’s own page. The [self-hosting vs API calculator](https://broadrouter.com/tools/self-hosting-vs-api-calculator/) does the whole break-even with your measured throughput and these GPU prices.

Free tools for this

-   [LLM API cost calculator](https://broadrouter.com/tools/llm-api-cost-calculator/) Price a workload on every model at once, with cache hits, reasoning and a flat-plan check.
-   [LLM token counter](https://broadrouter.com/tools/token-counter/) Exact token counts with each model's real tokenizer, with a token view and cost.
-   [Context window calculator](https://broadrouter.com/tools/context-window-calculator/) Will it fit? Window used per model, room left for output, cost to send.

[All free tools](https://broadrouter.com/tools/)

## Terms in this article

[Token](https://broadrouter.com/glossary/token/)

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

[Input and output tokens](https://broadrouter.com/glossary/input-and-output-tokens/)

Input tokens are the tokens a language model reads in a request, including the system prompt, conversation history, tool definitions and images. Output tokens are the tokens it generates in reply, including any reasoning. LLM APIs usually price them separately, and output usually costs more.

[Open-weight model](https://broadrouter.com/glossary/open-weight-model/)

An open-weight model is a machine learning model whose trained weights are published for anyone to download and run, usually under a licence with conditions. It differs from fully open-source AI, which also releases the training code and detailed information about the training data, so others can study and reproduce the model.

[Rate limit](https://broadrouter.com/glossary/rate-limit/)

A rate limit is a cap an API places on how much a client can use it in a time window, such as requests per minute or tokens per minute. Requests over the limit are refused, usually with HTTP 429 Too Many Requests, until usage falls back under it.

[Concurrency](https://broadrouter.com/glossary/concurrency/)

Concurrency is the number of requests a client has in flight at the same time. LLM APIs often cap it per key or plan, separately from requests per minute, because each open request holds GPU capacity until its response has finished.

[Throughput](https://broadrouter.com/glossary/throughput/)

Throughput, in LLM serving, is the total amount of work a system completes per unit of time across all concurrent requests, usually measured as output tokens per second or requests per second. It differs from per-stream speed, which is how fast a single request receives its tokens.

[Tokens per second](https://broadrouter.com/glossary/tokens-per-second/)

Tokens per second (TPS) is the rate at which a language model generates output tokens, usually measured for one request after its first token arrives. It sets how fast a reply streams once it starts. Summed across all concurrent requests it becomes system throughput; the wait before the first token is measured separately.

[Tool calling](https://broadrouter.com/glossary/tool-calling/)

Tool calling, also called function calling, is an LLM API feature in which the model returns a structured request to run a function the developer has defined, with JSON arguments. The application runs the function and sends the result back; the model never runs code itself.

## Questions

### Is it cheaper to self-host an LLM than to use an API?

Only at high, steady volume. A rented GPU bills every hour; an API bills only for tokens you use. If your GPU would sit idle most of the day, the API is cheaper even before you count engineering time.

### How much does it cost to rent an H100 per hour?

On RunPod's pricing page on 2026-11-14, an H100 SXM was USD 4.39 an hour on Secure Cloud and from USD 2.69 on Community Cloud. Lambda lists on-demand H100 SXM at USD 4.59 to 4.89 per GPU-hour, depending on node size, at prices in effect from 2026-11-16. Check the pages for current rates.

### How many GPUs do I need to self-host a large model?

At least enough memory for the weights plus the KV cache for your concurrent requests. Large mixture-of-experts models usually need a multi-GPU node. Check the model card for its parameter count and recommended setup.

### When does self-hosting make sense even if it costs more?

When data cannot leave your network, when you need a fine-tuned or custom model no API serves, or when you need guaranteed capacity. Those are control reasons, not cost reasons.

### How do I measure my GPU's real throughput?

Run your real prompts at your real concurrency with a serving engine such as vLLM or SGLang, and record total output tokens per second across all requests. Vendor peak numbers assume ideal batching.

## Keep reading

-   [Flat-rate vs pay-per-token LLM API: the break-even math](https://broadrouter.com/blog/flat-rate-vs-pay-per-token-llm-api/)
-   [LLM latency: TTFT vs tokens per second vs end-to-end time](https://broadrouter.com/blog/llm-latency-ttft-throughput/)
-   [What "unlimited" really means in an LLM API plan](https://broadrouter.com/blog/what-unlimited-llm-api-really-means/)
-   [OpenRouter vs a flat-rate plan for AI agents: which costs less?](https://broadrouter.com/blog/openrouter-alternative-flat-rate/)
-   [A private endpoint starts with a clear workload](https://broadrouter.com/blog/private-endpoint-planning/)
-   [Artificial Analysis alternatives: where to compare LLMs in 2026](https://broadrouter.com/blog/artificial-analysis-alternatives/)
-   [Live, on request, research: read a model catalogue clearly](https://broadrouter.com/blog/understand-model-status/)
-   [Guide: How the unlimited LLM API plans work for agents](https://broadrouter.com/guides/unlimited-llm-api-sessions/)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-self-hosting-llm-vs-api-cost&utm_content=page-cta) ·

**Sources**

-   [RunPod GPU pricing, Secure and Community Cloud hourly rates (fetched 2026-11-14)](https://www.runpod.io/pricing)
-   [Lambda Cloud pricing, on-demand per-GPU hourly rates (fetched 2026-11-16)](https://lambda.com/pricing)
-   [BroadNet Inference machine-readable prices (pricing.json) (checked 2026-11-15)](https://inference.broadnet.ai/pricing.json?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-self-hosting-llm-vs-api-cost&utm_content=sources)
-   [BroadNet Inference plans and typical speed (checked 2026-11-15)](https://inference.broadnet.ai/plans?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-self-hosting-llm-vs-api-cost&utm_content=sources)
-   [NVIDIA NIM benchmarking guide, throughput and latency metrics (checked 2026-11-16)](https://docs.nvidia.com/nim/benchmarking/llm/metrics)
