“Highly available” is a hope. An SLA is a number in a contract, with a defined remedy when the number is missed.
How it works
An SLA defines four things: what is covered, how downtime is measured, what you get when the target is missed, and how to claim. Google’s SLA for the Gemini online inference API on its Gemini Enterprise Agent Platform is a clear real example:
- Target. 99.5% Monthly Uptime Percentage for the
generateContentandstreamGenerateContentmethods (95% for models designated for shorter availability). - Downtime. “more than a five percent Error Rate”, where errors are HTTP 5XX responses to valid requests, counted only in periods of “five or more consecutive minutes”.
- Credits.
| Monthly uptime | Credit on the service’s monthly bill |
|---|---|
| 99.0% to under 99.5% | 10% |
| 95.0% to under 99.0% | 25% |
| Under 95.0% | 50% |
- Claiming. The customer “must notify Google technical support within 30 days” and provide log files showing the downtime periods. Credits are capped at 50% of the month’s bill for the service.
- Exclusions. Pre-GA features, Grounding with Google Search, errors from the customer’s own software, deadlines shorter than the server default, and quota errors are not covered.
The same SLA adds a latency target for some models under Provisioned Throughput, measured in output tokens per second. For Azure, Amazon Bedrock, OpenAI, Anthropic and others side by side, see LLM API SLAs compared.
Why it matters
- Uptime is not throughput. An SLA that counts only 5XX errors says nothing about rate limits (429) or slow responses, unless it has a latency target.
- Credits are small. A 10% credit rarely matches the cost of an outage to your product. Design for failure anyway: retries with backoff, timeouts and a fallback model or provider. See the errors and retries guide.
- Read the exclusions. Google’s SLA excludes pre-GA features and named services; check any SLA for similar carve-outs.
- Keep your own logs. Claims need evidence. Log request IDs, timestamps and status codes.
On BroadNet Inference
BroadNet’s self-serve API, both pay-as-you-go and plans, has no SLA. Its Terms of Service (§7.4) say: “We aim to keep the API available and publish what we measure, but self-serve has no availability commitment.” They add that “Service levels exist only in a separate written agreement” and that private endpoints and reserved capacity follow that agreement. If you need contracted uptime, ask BroadNet about a private endpoint; our private endpoint planning post covers what to plan for. For self-serve, plan a fallback: the provider checklist notes we found no SLA, status page or deprecation policy in BroadNet’s docs.