---
title: "BroadNet GPU hardware and serving, as BroadNet describes"
description: "What BroadNet says about the hardware behind BroadNet Inference: three GPU tiers from B300 to client appliances, throughput per GPU, and what its tests taught it."
url: https://broadrouter.com/blog/broadnet-gpu-fleet-and-serving/
lang: en
updated: 2026-11-15
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# BroadNet's GPU hardware and serving approach, as BroadNet describes it

By the BroadNet Inference team · 5 min read · Published 2026-11-15 · Updated 2026-11-16

BroadNet says it runs one serving stack on three hardware tiers: data-center GPUs from A100 to B300, a workstation tier of RTX and DGX Spark cards that carries the always-on public API, and appliances on client sites. It reports about 270 models run across 13 GPU families, NVIDIA and AMD. Some API models are served by US inference partners instead.

## Key takeaways

-   Three tiers, one stack: data center (B300, B200, H200, H100, A100), workstation (RTX PRO 6000, RTX 5090, RTX 3090, DGX Spark GB10, Radeon AI PRO R9700) and client site.
-   BroadNet says the workstation tier carries the always-on public API; data-center GPUs handle flagship serving, fine-tuning and private endpoints.
-   Its measured finding: NVIDIA CUDA turns about twice as much memory bandwidth into tokens as AMD ROCm on the same models.
-   Serving matters as much as weights: BroadNet reports the same Gemma 4 26B weights scored 0.13 lower at a hosted provider than on what it calls its own serving.
-   Not every API model runs on hardware BroadNet says is its own: BroadNet's terms say DeepSeek V4.1 Flash is partner-hosted.

**On this page**

1.  What hardware does BroadNet say it runs?
2.  Which API models run on it?
3.  How fast is each GPU?
4.  What does BroadNet say its hardware taught it?
5.  How is the Gemma 4 model served?
6.  What does this mean for you?

BroadNet describes its inference hardware in unusual detail. Its [model catalogue](https://www.broadnet.ai/inference/models) lists the GPUs it says it runs, measured throughput per card, and what it learned from running the same models across them. This page summarises what BroadNet says, in its words where it matters. We have not inspected the hardware, and the measurements are BroadNet’s own, from September 2026.

## What hardware does BroadNet say it runs?

BroadNet says it runs “one fleet, three tiers, one stack”: the same engines, quantisations and gateway from data-center accelerators down to appliances on client sites.

| Tier | GPUs BroadNet lists | What BroadNet uses it for |
| --- | --- | --- |
| Data center | B300, B200, H200, H100, A100 80 GB | Flagship serving, quantisation and calibration, fine-tuning, dedicated private endpoints, data-residency placements |
| Workstation | RTX PRO 6000 Blackwell, RTX 5090, RTX 3090, DGX Spark GB10, Radeon AI PRO R9700 | The always-on public API, benchmarking across CUDA and ROCm, model development |
| Client site | DGX Spark GB10, compact AMD appliances | On-premise and air-gapped deployments on the identical stack |

In total BroadNet reports about 270 models run, about 200 checkpoints loaded, about 40 model families, 12 or more quantisation formats and 13 GPU families. It also says “a single H100 serves the entire current public catalogue with headroom”.

## Which API models run on it?

Not all of them. BroadNet’s [privacy statement](https://www.broadnet.ai/inference-privacy) says its own models and the open-weight models it hosts run on BroadNet-owned or leased hardware in the United States, with partner capacity as backup, and that models served by partners run under zero-data-retention terms with US inference partners. Its [Terms of Service](https://www.broadnet.ai/terms) name [DeepSeek V4.1 Flash](https://broadrouter.com/models/deepseek-v4.1-flash/) as partner-hosted.

What BroadNet’s own pages say per model:

| Model on the API | Hardware BroadNet names |
| --- | --- |
| [Gemma 4 VL 26B](https://broadrouter.com/models/gemma-4-vl-26b/) | NVIDIA Blackwell-class GPUs with a hot backup |
| [Transcribe Arabic](https://broadrouter.com/models/cohere-transcribe-arabic/) and [Transcribe](https://broadrouter.com/models/cohere-transcribe/) | NVIDIA GPUs operated by BroadNet in the United States |
| DeepSeek V4.1 Flash | Partner-hosted (Terms of Service §7.3) |
| GLM 5.3, Qwen3.8 27B | Not stated for API traffic |

If where your data is processed matters, ask BroadNet before you send it. Its inference page says regional processing in MENA or Europe is available on request.

## How fast is each GPU?

BroadNet publishes aggregate output throughput per GPU for one model, Qwen3-VL 30B A3B Instruct at 32 concurrent requests:

| GPU | Tier | Memory bandwidth | Aggregate tokens/s |
| --- | --- | --- | --- |
| B300 | Data center | 8,000 GB/s | 14,500 |
| B200 | Data center | 8,000 GB/s | 14,000 |
| H200 | Data center | 4,800 GB/s | 8,400 |
| H100 | Data center | 3,350 GB/s | 5,800 |
| A100 80 GB | Data center | 2,039 GB/s | 3,500 |
| RTX PRO 6000 Blackwell | Workstation | 1,792 GB/s | 3,100 |
| RTX 3090 | Workstation | 936 GB/s | 1,630 |
| DGX Spark GB10 | Workstation | 273 GB/s | 1,597 |
| Radeon AI PRO R9700 | Workstation | 640 GB/s | 1,087 |

BroadNet notes that the data-center figures are “planning numbers from our measured per-GPU curve”, not direct measurements on each card. Most rows rise with memory bandwidth, the usual limit for decoding, but not all: DGX Spark GB10 (273 GB/s) nearly matches the RTX 3090 (936 GB/s), and the Radeon AI PRO R9700 (640 GB/s) trails both.

## What does BroadNet say its hardware taught it?

The catalogue lists eight findings, each marked “measured on our own hardware”. The ones that matter most for API buyers:

-   **CUDA vs ROCm.** Running the same models on NVIDIA CUDA and AMD ROCm, CUDA turned about twice as much memory bandwidth into tokens, so BroadNet places workloads accordingly.
-   **Speculative decoding under load.** EAGLE3 cost 69% of throughput at 32 concurrent requests, so BroadNet kept multi-token-prediction drafters. Its [Uno serving study](https://broadrouter.com/blog/uno-in-vllm-serving-study/) measures the same effect for another method.
-   **Serving matters as much as weights.** The same Gemma 4 26B weights served by a hosted provider scored 0.13 lower on average than what BroadNet calls “our own serving”. BroadNet’s rule: keep a challenger and its baseline on the same serving before comparing them.
-   **A smaller active model can win.** The 26B mixture of experts beat the dense Gemma 4 31B on 5 of 6 pipeline steps with about one eighth of the active parameters.
-   **Frontier parity on one pipeline.** In a blind evaluation of 240 traces, BroadNet’s served Gemma 4 matched Claude Sonnet on five of six measured steps, with 1.2 to 6.5 times faster responses per step.
-   **Per-request speed.** Qwen3-VL 30B A3B ran at 36.5 to 110 tokens per second per request on what BroadNet describes as its own serving, by BroadNet’s measurement, against 7 to 19 at four OpenRouter providers BroadNet measured.
-   **Architecture beats model choice.** On a hard set where two local vision models disagreed, escalating to a hosted adjudicator moved accuracy from 1 of 12 to 11 of 12.

The Sonnet comparison is the strongest claim, and it is narrow: one company’s chatbot pipeline, judged blind by three judges, on 240 production conversations from 11 tenants, per BroadNet’s [Gemma 4 page](https://www.broadnet.ai/inference/models/gemma-4-vl-26b). It does not mean Gemma 4 matches Sonnet in general. BroadNet has not published a full write-up of that evaluation on its research page.

## How is the Gemma 4 model served?

BroadNet’s Gemma 4 VL 26B is the clearest example of its serving choices, per its model page:

-   **Weights:** Google’s Gemma 4 26B A4B, quantised by BroadNet to 4-bit AWQ. BroadNet tested NVFP4 and retired it after a head-to-head.
-   **Decoding:** multi-token-prediction speculative decoding with two drafted tokens (k=2), and an fp8 [KV cache](https://broadrouter.com/glossary/kv-cache/).
-   **Template:** a chat template BroadNet tuned for its conversational workflow.
-   **Resilience:** a hot backup with automatic failover.
-   **Result:** about 136 tokens per second on a single stream, and 90.86 on BroadNet’s internal quality battery.

See [quantisation](https://broadrouter.com/glossary/quantization/) and [speculative decoding](https://broadrouter.com/glossary/speculative-decoding/) for the terms.

## What does this mean for you?

For a pay-as-you-go user, BroadNet’s serving choices show up as the published speed, about 100 tokens per second typical on the plan model, and in the limits: 30 requests a minute and 4 in flight per key. BroadNet’s terms say self-serve has no availability commitment; service levels come only with a written agreement. [How other providers’ SLAs compare](https://broadrouter.com/blog/llm-api-sla-uptime-compared/).

For larger needs, BroadNet offers private endpoints on reserved GPUs, with your model, key, limits and region, and on-premise appliances on the same stack. If you are weighing that against renting your own GPUs, our [self-hosting vs API cost guide](https://broadrouter.com/blog/self-hosting-llm-vs-api-cost/) works through the arithmetic, and [the models BroadNet ruled out](https://broadrouter.com/blog/models-broadnet-ruled-out/) shows how it chose what to serve.

Free tools for this

-   [OpenAI to BroadNet migration helper](https://broadrouter.com/tools/openai-to-broadnet-migration/) Paste OpenAI SDK code; get base URL, key and model edits, a diff and the checks that remain.
-   [max\_tokens calculator for reasoning models](https://broadrouter.com/tools/reasoning-token-budget/) Answer length + reasoning budget + margin = a max\_tokens that does not cut the answer off.
-   [JSON schema generator for LLM structured output](https://broadrouter.com/tools/json-schema-generator/) Sample JSON to strict schema, Pydantic, Zod and a ready request.
-   [LLM JSON output validator](https://broadrouter.com/tools/llm-json-output-validator/) Parse errors by line and column, safe repairs, cut-off detection and JSON Schema checks for model output.

[All free tools](https://broadrouter.com/tools/)

## Terms in this article

[Token](https://broadrouter.com/glossary/token/)

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

[Concurrency](https://broadrouter.com/glossary/concurrency/)

Concurrency is the number of requests a client has in flight at the same time. LLM APIs often cap it per key or plan, separately from requests per minute, because each open request holds GPU capacity until its response has finished.

[Pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/)

Pay-as-you-go is a pricing model in which you pay only for what you use, metered per unit such as tokens, requests or audio minutes, with no subscription or minimum commitment. Most LLM APIs bill this way, from a prepaid balance or a periodic invoice.

[Speculative decoding](https://broadrouter.com/glossary/speculative-decoding/)

Speculative decoding is an inference technique in which a small, fast draft model proposes several next tokens and the large target model verifies them in a single forward pass, keeping the accepted ones. It speeds up generation without changing the distribution of the output.

[Throughput](https://broadrouter.com/glossary/throughput/)

Throughput, in LLM serving, is the total amount of work a system completes per unit of time across all concurrent requests, usually measured as output tokens per second or requests per second. It differs from per-stream speed, which is how fast a single request receives its tokens.

[Tokens per second](https://broadrouter.com/glossary/tokens-per-second/)

Tokens per second (TPS) is the rate at which a language model generates output tokens, usually measured for one request after its first token arrives. It sets how fast a reply streams once it starts. Summed across all concurrent requests it becomes system throughput; the wait before the first token is measured separately.

[AWQ (activation-aware weight quantization)](https://broadrouter.com/glossary/awq/)

AWQ (activation-aware weight quantization) is a post-training method that compresses an LLM's weights to low precision, usually 4-bit integers. It uses activation statistics from a small calibration set to find the most important weight channels and scales them so they lose less accuracy. It needs no backpropagation.

[Chat template](https://broadrouter.com/glossary/chat-template/)

A chat template is the formatting rule, usually written in Jinja and shipped with a model's tokenizer, that converts a list of chat messages with roles into the single token sequence the model expects, including its special control tokens. Using the wrong template can sharply degrade a model's answers.

## Questions

### Does BroadNet run its own GPUs?

BroadNet says it operates its own NVIDIA and AMD GPU fleet. Its privacy statement says its own models and the open-weight models it hosts run on BroadNet-owned or leased hardware in the United States, with partner capacity as backup. Partner-served models, such as DeepSeek V4.1 Flash, run with US inference partners.

### Which GPU serves the public API?

BroadNet's catalogue places "the always-on public API" on its workstation tier (RTX PRO 6000 Blackwell, RTX 5090, RTX 3090, DGX Spark GB10, Radeon AI PRO R9700). Its Gemma 4 page names NVIDIA Blackwell-class GPUs with a hot backup.

### Is there a published uptime SLA?

No for self-serve. BroadNet's Terms of Service say self-serve has no availability commitment, and service levels exist only in a separate written agreement, such as for a private endpoint.

### Can I get a dedicated GPU?

Yes. BroadNet offers private endpoints on a reserved GPU or node, with your choice of model, your own key and limits, and the region you need. They are priced to your traffic on request.

### Can BroadNet run models on my premises?

BroadNet says its client-site tier runs the same stack on DGX Spark GB10 and compact AMD appliances for on-premise and air-gapped deployments.

## Keep reading

-   [Models BroadNet evaluated and ruled out, and why](https://broadrouter.com/blog/models-broadnet-ruled-out/)
-   [Uno in vLLM: what BroadNet's serving study measured](https://broadrouter.com/blog/uno-in-vllm-serving-study/)
-   [Uno on Gemma 4 26B: BroadNet's mixture-of-experts study](https://broadrouter.com/blog/uno-mixture-of-experts-gemma-4-study/)
-   [Self-hosting an LLM vs an API: the real cost and break-even](https://broadrouter.com/blog/self-hosting-llm-vs-api-cost/)
-   [LLM latency: TTFT vs tokens per second vs end-to-end time](https://broadrouter.com/blog/llm-latency-ttft-throughput/)
-   [A private endpoint starts with a clear workload](https://broadrouter.com/blog/private-endpoint-planning/)
-   [Before fine-tuning: define the problem to solve](https://broadrouter.com/blog/fine-tuning-readiness/)
-   [Guide: Try BroadNet Inference in your browser, no account](https://broadrouter.com/guides/try-broadnet-inference-playground/)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-broadnet-gpu-fleet-and-serving&utm_content=page-cta) ·

**Sources**

-   [BroadNet model catalogue: capacity, throughput per GPU and hardware findings (measured September 2026; fetched 2026-11-15)](https://www.broadnet.ai/inference/models)
-   [BroadNet Gemma 4 VL 26B model page: serving configuration and blind evaluation (fetched 2026-11-15)](https://www.broadnet.ai/inference/models/gemma-4-vl-26b)
-   [BroadNet Inference privacy statement: where processing happens (fetched 2026-11-15)](https://www.broadnet.ai/inference-privacy)
-   [BroadNet Terms of Service, §7.3 and §7.4: partner hosting and availability (fetched 2026-11-15)](https://www.broadnet.ai/terms)
-   [BroadNet Research: open findings from its lab (fetched 2026-11-15)](https://www.broadnet.ai/research)
