---
title: "What is time to first token (TTFT) in LLM APIs? | BroadRouter"
description: "Time to first token (TTFT) is the wait between sending an LLM request and receiving the first output token. What makes it up: queueing, prefill and network."
url: https://broadrouter.com/glossary/time-to-first-token/
lang: en
updated: 2026-11-16
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Time to first token (TTFT)

Updated 2026-11-16

Time to first token (TTFT) is the time from sending a request to a language model until the first output token arrives. It includes network time, queueing on the server and prefill, the step where the model reads the whole prompt, so it grows with prompt length and load.

Also called: TTFT, first-token latency, time to first content token, first token time

TTFT is the pause before anything happens. In a chat window it is the blank screen; in a voice bot it is dead air.

## How it works

NVIDIA’s benchmarking guide breaks TTFT into three parts: “request queuing time, prefill time, and network latency.” Each has its own cause:

| Part | What drives it | What you can change |
| --- | --- | --- |
| Network | Distance, TLS setup, proxies | Reuse connections; call from a nearby region |
| Queueing | How busy the server is | Off-peak timing, a less loaded model or tier |
| Prefill | Prompt length, model size | Shorter prompts, stable cacheable prefixes |

Prefill is usually the big one for long inputs. The model runs the whole prompt through every layer to build the [KV cache](https://broadrouter.com/glossary/kv-cache/) before it can produce token one. NVIDIA notes that on a busy server one request’s prefill can overlap with another’s generation, so TTFT also moves with other people’s traffic.

Two definitions matter when you read numbers. NVIDIA’s AIPerf counts the first token only if it carries content. For streaming text in the Responses API, OpenAI’s guide times TTFT “until you receive the first nonempty response.output\_text.delta”. With [reasoning tokens](https://broadrouter.com/glossary/reasoning-tokens/), the first visible word can come much later than the first generated token, so say which one you mean.

## Why it matters

TTFT sets how fast a product feels, more than raw speed does. OpenAI’s guide says “a long prompt can delay the first token, especially when much of the input is new”. This is where [cached input tokens](https://broadrouter.com/glossary/cached-input-tokens/) help. The same guide says prompt caching “can reuse an eligible exact prefix match” and suggests comparing TTFT with and without cache hits. A cached prefix skips part of prefill, so a repeated system prompt costs less and can start sooner.

For agents, TTFT is paid on every step. Twenty tool calls with a two-second TTFT add forty seconds of waiting before counting any output.

TTFT is one part of end-to-end [latency](https://broadrouter.com/glossary/latency/). The rest is generation time, set by [tokens per second](https://broadrouter.com/glossary/tokens-per-second/).

## On BroadNet Inference

BroadNet’s docs describe [streaming](https://broadrouter.com/glossary/streaming/) with Server-Sent Events, ending with `data: [DONE]`, which is how you observe the first token on the client. DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason before they answer, so time to first visible text includes thinking. BroadNet does not publish a TTFT figure. For a timing script and p50/p95 reporting, see [LLM latency: TTFT vs tokens per second](https://broadrouter.com/blog/llm-latency-ttft-throughput/).

## Questions

### Why is TTFT slow with a long prompt?

The model has to read every prompt token and build its KV cache before it can write anything. NVIDIA's metrics guide says longer prompts increase TTFT for exactly this reason.

### Does streaming reduce time to first token?

No. Streaming lets you see the first token as soon as it exists; without it you wait for the whole reply. The server-side work before that token is the same either way.

### Does an empty first chunk count as the first token?

Not in most benchmarks. NVIDIA's AIPerf ignores initial responses with no content, and OpenAI times TTFT to the first non-empty text delta.

## See also

-   [Latency](https://broadrouter.com/glossary/latency/) (term)
-   [Tokens per second](https://broadrouter.com/glossary/tokens-per-second/) (term)
-   [KV cache](https://broadrouter.com/glossary/kv-cache/) (term)
-   [Streaming](https://broadrouter.com/glossary/streaming/) (term)
-   [Cached input tokens](https://broadrouter.com/glossary/cached-input-tokens/) (term)
-   [Speculative decoding](https://broadrouter.com/glossary/speculative-decoding/) (term)
-   [Prefill and decode](https://broadrouter.com/glossary/prefill-and-decode/) (term)
-   [Stream chat completions and handle dropped connections](https://broadrouter.com/guides/streaming-chat-completions/) (guide)
-   [Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/) (guide)
-   [LLM latency: TTFT vs tokens per second vs end-to-end time](https://broadrouter.com/blog/llm-latency-ttft-throughput/) (article)
-   [LLM streaming explained: SSE chunks, deltas, usage and cancel](https://broadrouter.com/blog/llm-streaming-sse-explained/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-time-to-first-token&utm_content=page-cta) ·

**Sources**

-   [NVIDIA NIM, LLM benchmarking metrics (time to first token)](https://docs.nvidia.com/nim/benchmarking/llm/latest/metrics.html)
-   [OpenAI, latency optimization guide](https://developers.openai.com/api/docs/guides/latency-optimization)
-   [BroadNet Inference API reference, streaming and reasoning](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-time-to-first-token&utm_content=sources)
