BroadRouter
Get started

GPU-hour

Updated

A GPU-hour is one graphics processor used for one hour. Cloud providers bill rented GPU instances by the hour or second, and researchers use GPU-hours to state how much compute a job took. You pay for the machine's time, whether or not it is busy.

Also called: GPU hours, GPU-hours, GPU instance hour, GPU rental hour

A GPU-hour is the unit you pay in when you rent hardware. A per-token API hides it; a self-hosted model makes it the whole bill.

How it works

Cloud providers price GPU instances per hour, then meter more finely. AWS’s on-demand page says pricing is “per instance-hour consumed”, with partial hours on Linux billed per second and a 60-second minimum. Google Cloud charges “all vCPUs, GPUs, and GB of memory” a 1-minute minimum, then 1-second increments. An instance with 8 GPUs running for 3 hours is 24 GPU-hours.

The meter runs while the instance is up, not while it is useful. An idle GPU at 3 a.m. costs the same as a busy one.

Researchers use the same unit for compute. The GPTQ paper, for example, quantizes a 175B-parameter model “in approximately four GPU hours”.

Why it matters

To compare a GPU-hour with per-token pricing, convert through throughput:

cost per 1M output tokens = hourly GPU cost ÷ (tokens per second × 3,600 ÷ 1,000,000)

The catch is the denominator. Throughput depends on the model, precision, batch size and how busy you keep the GPU. A rented GPU at 20% load costs five times as much per token as the same GPU at full load. That is the main reason teams with uneven traffic use APIs; see self-hosting vs API cost.

On BroadNet Inference

BroadNet does not rent GPUs by the hour. It sells API access to its models; its Terms say they run on BroadNet-owned or leased hardware in the United States, or with US inference partners. Two of its options are priced per hour, and neither is a GPU-hour:

GPU-hour (cloud rental) BroadNet hourly key Meteor plan
What you get A whole GPU and the machine around it One stream on a shared model One session on a shared model
Model Anything that fits BroadNet Gemma 4 VL 26B only DeepSeek V4.1 Flash only
Limits (per BroadNet’s docs) The hardware 120 requests/min, 300,000 tokens/min, 1 stream One request at a time, no token meter
Price Set by the cloud $0.50 per hour $1 per hour per session

You do not manage drivers, scaling or idle time with either BroadNet option, but you also cannot choose the hardware or load a different model. Running the hourly key all month would be $0.50 × 24 × 30 = $360.00. For details, see hourly LLM API keys explained and the flat-rate plan entry.

Questions

Is a BroadNet hourly key a GPU-hour?

No. An hourly key buys one hour of one request stream on a shared model, BroadNet Gemma 4 VL 26B, with fixed limits. You do not get a GPU, a choice of model or any control over the server.

Are GPU-hours billed for the full hour?

Often not. AWS bills many Linux instances per second after a 60-second minimum, and Google Cloud charges GPUs a 1-minute minimum, then per second. Check the instance type and operating system.

How do I turn a GPU-hour price into a per-token price?

Divide the hourly cost by the millions of tokens the GPU produces in an hour, which is tokens per second × 3,600 ÷ 1,000,000. Measure throughput at your real load, because idle time is billed too.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity