BroadRouter
Get started

LLM API usage logging: cost per user and feature, PII redacted

By the BroadNet Inference team · 4 min read · Published

Wrap every LLM call in one function that records model, feature, a hashed user ID, prompt, cached, reasoning and output tokens, cost, latency, status and the provider request ID. Compute cost from a price table, not the invoice. Log metadata by default; if you must keep prompt text, redact it before writing and set a short retention.

Provider dashboards tell you what you spent. They rarely tell you which feature or which customer spent it, and they cannot tell you why a bill jumped on Tuesday. Your own usage log can. This post shows the log schema, a wrapper that fills it, how to compute cost per call, and how to keep personal data out. What to do with the numbers is covered elsewhere: cost per user and SaaS pricing and budgets and loop guards.

What should you log for each LLM call?

One row per attempt, retries included:

Field Example Why
ts 2026-11-14T09:12:03Z Time series, incident timelines
feature support_reply Cost per feature
user_hash hmac:4f1c… Cost per user without storing identity
tenant acme Cost per customer in B2B apps
model broadnet/deepseek-v4.1-flash Price lookup, model comparisons
prompt_tokens 18,240 Input cost
cached_tokens 16,384 Cached input is cheaper; track the cache rate
reasoning_tokens 612 Hidden output cost on reasoning models
completion_tokens 1,104 Output cost (includes reasoning)
cost_usd computed The number finance asks for
latency_ms, ttft_ms 4,210 / 690 Performance regressions
status 200, 429, 503 Error and retry rates
request_id X-Request-ID value Support tickets and invoice disputes
prompt_version support-v14 Ties cost changes to prompt changes

No prompt text, no response text. Those come later, and only with redaction.

How do you compute cost per call?

Read the usage block and price each token type separately. BroadNet publishes machine-readable prices in pricing.json, with per-1M prices for input, cached input and output per model:

import json, urllib.request

PRICES = {m["id"]: m for m in json.load(urllib.request.urlopen("https://inference.broadnet.ai/pricing.json"))["models"]}

def call_cost(model: str, usage) -> float:
    p = PRICES[model]
    details = getattr(usage, "prompt_tokens_details", None)
    cached = (getattr(details, "cached_tokens", 0) or 0) if details else 0
    new_input = usage.prompt_tokens - cached
    return (
        new_input * (p.get("input_per_m") or 0)
        + cached * (p.get("cached_input_per_m") or p.get("input_per_m") or 0)
        + usage.completion_tokens * (p.get("output_per_m") or 0)
    ) / 1_000_000

Cache the price file and refresh it daily rather than fetching it on every call. Two easy mistakes: pricing cached tokens as new input (overstates cost), and forgetting that reasoning tokens are already inside completion_tokens (do not add them twice).

What does the logging wrapper look like?

import hashlib, hmac, os, time, logging
from openai import OpenAI

log = logging.getLogger("llm.usage")
client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])
HASH_KEY = os.environ["USAGE_HASH_KEY"].encode()

def user_hash(user_id: str) -> str:
    return "hmac:" + hmac.new(HASH_KEY, user_id.encode(), hashlib.sha256).hexdigest()[:16]

def chat(*, feature: str, user_id: str, prompt_version: str, **kwargs):
    t0 = time.perf_counter()
    row = {"feature": feature, "user_hash": user_hash(user_id),
           "model": kwargs["model"], "prompt_version": prompt_version}
    try:
        raw = client.chat.completions.with_raw_response.create(**kwargs)
        resp = raw.parse()
        u = resp.usage
        d = getattr(u, "completion_tokens_details", None)
        row.update(
            status=200,
            request_id=raw.headers.get("x-request-id"),
            prompt_tokens=u.prompt_tokens,
            cached_tokens=getattr(getattr(u, "prompt_tokens_details", None), "cached_tokens", 0) or 0,
            reasoning_tokens=(getattr(d, "reasoning_tokens", 0) or 0) if d else 0,
            completion_tokens=u.completion_tokens,
            cost_usd=round(call_cost(kwargs["model"], u), 6),
        )
        return resp
    except Exception as e:
        row.update(status=getattr(e, "status_code", "error"),
                   request_id=getattr(getattr(e, "response", None), "headers", {}).get("x-request-id"))
        raise
    finally:
        row["latency_ms"] = int((time.perf_counter() - t0) * 1000)
        log.info("llm_call", extra={"llm": row})

with_raw_response gives you the headers, so you can keep the X-Request-ID. For streamed calls, add stream_options={"include_usage": True} and fill the row from the final chunk, as shown in the SSE streaming post. Keep HASH_KEY secret and stable: the same user always maps to the same hash, but the hash cannot be reversed without the key.

How do you attribute cost to users and features?

With the rows above in any SQL store, attribution is one query:

SELECT feature, model,
       count(*)                          AS calls,
       sum(cost_usd)                     AS cost,
       sum(cached_tokens)::float / nullif(sum(prompt_tokens), 0) AS cache_rate,
       avg(completion_tokens)            AS avg_out
FROM llm_calls
WHERE ts >= now() - interval '7 days'
GROUP BY feature, model
ORDER BY cost DESC;

Swap feature for user_hash or tenant for per-customer cost. Three follow-ups pay off fast:

  1. Top 1% of users by cost. Often a single loop or a bot. A falling cache_rate is the other early warning; the prompt caching calculator shows what each point of cache rate is worth.
  2. Cost per successful outcome, not per call: join to your own success events.
  3. Retry share. Rows with status 429 or 5xx show how much you spend on failures; see handling 429 errors.

How do you keep PII out of LLM logs?

The safest log has no prompt text at all. When you need text, for debugging or evals, follow these steps in order:

  1. Sample. Keep text for a small share of calls and for failures, not for everything.
  2. Redact before writing. Run redaction in the process that writes the log. Redacting later means the raw text already sits on disk and in backups.
  3. Layer the detectors. Regex for structured data, an NER tool for names and addresses.
  4. Store separately. Put text in a different table or bucket with stricter access and a short retention, linked by request_id.
  5. Never log secrets. The OWASP logging cheat sheet lists access tokens, passwords and encryption keys among data to exclude; the API key belongs on that list.

A minimal regex layer:

import re
PATTERNS = [
    (re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+"), "[EMAIL]"),
    (re.compile(r"\+?\d[\d\s().-]{7,}\d"), "[PHONE]"),
    (re.compile(r"\b(?:\d[ -]?){13,19}\b"), "[CARD]"),
    (re.compile(r"\b(sk|key|bearer)[-_ ]?[A-Za-z0-9]{16,}\b", re.I), "[SECRET]"),
]
def redact(text: str) -> str:
    for pat, label in PATTERNS:
        text = pat.sub(label, text)
    return text

This catches the obvious cases and misses names, addresses and anything unusual. Microsoft Presidio adds entity recognition for those, and its README says plainly that there is no guarantee it will find all sensitive information. Plan for misses: short retention and tight access limit the damage when one gets through.

Where BroadNet Inference fits

BroadNet Inference returns the usage block with cached and reasoning tokens when the model reports them, an X-Request-ID on every response, and a public price file you can load for cost computation. On its side, prompts and answers are processed for the request and not retained; it keeps only content-free records such as timings, token counts and cost (how zero data retention works). That makes your own logs the only copy of any prompt text, so the redaction rules above decide your real retention. BroadNet’s chat requests have no per-user metadata field, so attribution lives in your wrapper. For separate budgets per team, use separate keys; the keys guide covers labels and allow-lists.

Questions

How do I track LLM API cost per user?

Tag each call with a user identifier, read the usage block from the response, multiply token counts by your price table, and write one log row per call. Group by the user tag. Use a hashed ID rather than an email.

Should I log full prompts and responses?

Not by default. They are where personal data lives. Log token counts and metadata for every call, and keep full text only for sampled or failed calls, after redaction, with a short retention period.

Why does my computed cost differ from the provider invoice?

Usually because cached tokens were priced as new input, reasoning tokens were left out of output, or retries were not logged. Log every attempt, including failed ones that returned usage.

How do I get token usage from a streaming response?

Set stream_options include_usage to true. The final chunk then carries the usage block with an empty choices list.

What PII redaction tool should I use for LLM logs?

Start with regex for structured data such as emails, phone numbers and card numbers, and add an NER-based tool such as Microsoft Presidio for names and addresses. Presidio's own README says it cannot guarantee it finds everything.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity