A GPU-hour is the unit you pay in when you rent hardware. A per-token API hides it; a self-hosted model makes it the whole bill.
How it works
Cloud providers price GPU instances per hour, then meter more finely. AWS’s on-demand page says pricing is “per instance-hour consumed”, with partial hours on Linux billed per second and a 60-second minimum. Google Cloud charges “all vCPUs, GPUs, and GB of memory” a 1-minute minimum, then 1-second increments. An instance with 8 GPUs running for 3 hours is 24 GPU-hours.
The meter runs while the instance is up, not while it is useful. An idle GPU at 3 a.m. costs the same as a busy one.
Researchers use the same unit for compute. The GPTQ paper, for example, quantizes a 175B-parameter model “in approximately four GPU hours”.
Why it matters
To compare a GPU-hour with per-token pricing, convert through throughput:
cost per 1M output tokens = hourly GPU cost ÷ (tokens per second × 3,600 ÷ 1,000,000)
The catch is the denominator. Throughput depends on the model, precision, batch size and how busy you keep the GPU. A rented GPU at 20% load costs five times as much per token as the same GPU at full load. That is the main reason teams with uneven traffic use APIs; see self-hosting vs API cost.
On BroadNet Inference
BroadNet does not rent GPUs by the hour. It sells API access to its models; its Terms say they run on BroadNet-owned or leased hardware in the United States, or with US inference partners. Two of its options are priced per hour, and neither is a GPU-hour:
| GPU-hour (cloud rental) | BroadNet hourly key | Meteor plan | |
|---|---|---|---|
| What you get | A whole GPU and the machine around it | One stream on a shared model | One session on a shared model |
| Model | Anything that fits | BroadNet Gemma 4 VL 26B only | DeepSeek V4.1 Flash only |
| Limits (per BroadNet’s docs) | The hardware | 120 requests/min, 300,000 tokens/min, 1 stream | One request at a time, no token meter |
| Price | Set by the cloud | $0.50 per hour | $1 per hour per session |
You do not manage drivers, scaling or idle time with either BroadNet option, but you also cannot choose the hardware or load a different model. Running the hourly key all month would be $0.50 × 24 × 30 = $360.00. For details, see hourly LLM API keys explained and the flat-rate plan entry.