BroadRouter
Get started

How to test an LLM for sensitive-data safety: BroadNet's method

By the BroadNet Inference team · 3 min read · Published

BroadNet's April 2026 preprint (DOI 10.5281/zenodo.19574049, also on SSRN) sets out a test for sensitive-data safety before deployment. Test four categories (credentials, PII, health and financial data) separately, score leaks and misses as different failures, use binary scoring, and run every test at least three times. Prompt rules and fine-tuning on negative examples made leaks worse in its tests.

Cards reading Name and Email turning into the placeholders [PERSON] and [EMAIL], with the text "Redact. Process. Restore." and "LSDF by BroadNet".

BroadNet’s research arm published a practical method for testing whether a large language model leaks sensitive data before you deploy it. The preprint, How to Evaluate an LLM for Sensitive Data Safety Before Deploying It, is on Zenodo under DOI 10.5281/zenodo.19574049, dated 14 April 2026, and mirrored on SSRN. Its Zenodo record lists Mohammad Al Zubaidi as the author.

This page summarises the method and its findings. It is BroadNet’s work, and the findings are its own; we have not re-run them.

Why don’t normal benchmarks catch this?

Public benchmarks measure capability: what a model can do at its best. A model deployed to audit a configuration file or a clinical note must also not do something: repeat the secret it found. The paper’s example makes the point. A model that scores 95 on 99 tasks and echoes an API key on the hundredth averages 94 and is still unusable in production.

BroadNet calls this testing for floors rather than ceilings, and the paper treats “capability and constraint” as independent axes.

What does the method test?

Four categories, each tested on its own:

Category Examples How a model recognises it
Credentials API keys, tokens, SSH keys, database passwords Mostly by format; passwords in connection strings have no format cue
Personally identifiable information (PII) Names, emails, phone numbers, government IDs Format for structured IDs, meaning for free text
Protected health information (PHI) Diagnoses, prescriptions, clinical notes Often by context: a diagnosis next to a name
Financial data Account numbers, balances, transactions Format, but less often treated as sensitive

The abstract reports testing “24+ models across 6 model families” and finds a clear hierarchy: format-based recognition, such as structured credentials and Social Security numbers, is much more reliable than context-based recognition, such as names that become sensitive next to a diagnosis. One model with a 0% credential leak rate leaked patient identifiers on every PHI test run.

How should results be scored?

The paper proposes a minimum protocol:

  1. Binary scoring. Each run is safe or not; no partial credit for a “mostly safe” answer.
  2. Leak and miss scored separately. Collapsing them into one number hides the one that matters for your use case.
  3. Multiple runs. Three or more per model and probe, because the same model can pass one day and leak the next.
  4. Category by category. Safety on credentials says nothing about safety on health records.

BroadNet’s practitioner write-up adds operating rules it uses on pilots: use synthetic data only, test the full deployment configuration (system prompt, tools and retrieval included), require five consecutive clean runs before production, and re-test monthly for self-hosted models and weekly for third-party models that update often.

What did not work?

Two common fixes made things worse in BroadNet’s tests:

  • Prompt instructions. Adding “do not reveal credentials” was sometimes ignored, and the most forceful prompting strategy caused leaks in a model that was clean at baseline.
  • Fine-tuning on negative examples. Training a model to refuse a category increased leak rates rather than lowering them.

What held up, in BroadNet’s account, was three layers together: choose a lower-risk model for the category, filter the output with a deterministic tool (it names Microsoft Presidio and LLM Guard), and redact input before the prompt is built when the task does not need the values. Our guide to sending sensitive data to an LLM API shows the redaction step in code.

What does the paper not publish?

Under its responsible publication policy, BroadNet publishes categories, scoring philosophy, sample sizes and caveats, but not exact prompt templates, credential formats or pass and fail thresholds. Its reason: publishing them would lower the effort needed for extraction with no defence available to deployers. Model providers and safety researchers can request them. That makes the method reproducible in shape, but not number for number.

How does this connect to BroadNet Inference?

This is a test of models, not of a provider. It applies to any model on any API, including the ones on BroadNet Inference. The companion paper, the Safety Map, applies it to 39 models from 14 labs and finds that sensitive values can also leave through tool-call arguments and reasoning fields. BroadNet later released an open-source firewall built on both papers, LSDF, described in the Safety Map summary.

Provider retention is a separate question: zero data retention covers what BroadNet stores, not what a model writes into your own systems.

Questions

What is the difference between a leak and a miss?

A leak is the model repeating a sensitive value where it should not, for example echoing an API key into a report. A miss is the model failing to flag a sensitive value when that is its job. BroadNet's method scores them separately because they carry different risks.

How many runs does the test need?

At least three per probe. BroadNet's practitioner protocol adds five consecutive clean runs before it calls a model safe for production, spread over at least two calendar days to catch provider-side changes.

Does a system prompt saying "never reveal secrets" help?

Not reliably. In BroadNet's tests the most forceful prompting strategy caused leaks in a model that was clean without it, and fine-tuning on negative examples also raised leak rates.

Can I get BroadNet's exact test prompts?

Not publicly. Under its responsible publication policy BroadNet withholds exact prompt templates, credential formats and pass or fail thresholds, and shares them with model providers and safety researchers on request.

Does zero data retention make this test unnecessary?

No. Zero data retention controls what the provider stores. This test is about what the model writes into its own output, which then lands in your logs, tickets and downstream systems.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity