The Safety Map is BroadNet’s survey of how large language models handle sensitive data. The preprint, The Safety Map: What Does and Doesn’t Transfer in LLM Sensitive-Data Handling, was published on Zenodo on 22 April 2026 under DOI 10.5281/zenodo.19688433, with Mohammad Al Zubaidi as author. It is the companion to BroadNet’s sensitive-data evaluation method.
What follows is our summary of the abstract and BroadNet’s research page. The verdicts are BroadNet’s.
What did the survey cover?
| Dimension | Coverage |
|---|---|
| Models | 39, from 14 independent labs |
| Probe scenarios | 7 |
| Categories | Credentials, PII, protected health information (PHI), financial account numbers, data-loss-prevention (DLP) scanning |
| Evidence | 135+ classified model-probe combinations from about 1,300 multi-run evaluations |
BroadNet’s catalogue says it has pushed about 90 remote and API models, which it never ran on its own hardware, through its safety, routing and Arabic test harnesses, and that those findings feed this map.
What are the main findings?
Safety is a surface, not a single score. It varies along three axes that stack:
- Architecture. The paper reports a capacity floor of about 24B active parameters on credentials, across routing designs, with small-active mixture-of-experts models as the main case.
- Generation. Alignment quality moves the boundary within a model family.
- The lab’s alignment investment. A flagship label from a well-known lab is not a guarantee.
None of the three predicts safety on its own; they combine.
Safety does not transfer between categories. Handling credentials and Social Security numbers cleanly did not carry over to names, health records or account numbers in the same model.
Output channels leak independently. This is the finding with the most practical weight:
- Of 73 runs that filled
tool_calls.arguments, 20 sent sensitive values through the tool call. Among the models behind those runs was Moonshot’s Kimi K2.6, tested on its release day, a model the paper rates otherwise multi-run safe on credentials, PII, financial data and DLP. - Of 43 runs in which the provider filled
reasoning_contentwith at least 20 characters, 29 leaked sensitive values into the reasoning while the chat reply was rated safe, missed or truncated.
Pruning can break safety. Two independent pruned versions of Meta’s Llama line, made by NVIDIA with neural architecture search, broke health-data safety through different output channels.
The paper ends with a five-tier ranking. Its top cluster, uniformly safe on every category tested, is Claude Opus 4.7, GPT-5.4 and GPT-5.4-mini, Gemini 3 Flash and Gemini 3.1 Pro. The abstract says xAI Grok and Meta Llama flagships do not make that cluster.
What should you change in your own stack?
- Filter every channel. A check on the visible reply misses tool-call arguments and reasoning fields. If you use tool calling or a reasoning model, inspect both.
- Test by category. A clean credential result says nothing about health or salary data.
- Retest on model updates. The Kimi K2.6 result came from a day-of-release model.
- Redact first where you can. A model cannot leak what it never saw. Our sensitive data guide shows placeholder redaction in Python.
What is LSDF?
LSDF, the LLM Sensitive Data Firewall, is the control BroadNet built from these two papers and released as open source under Apache-2.0 at github.com/brntech/lsdf. It is a reverse proxy for the OpenAI-compatible chat completions API: you point your client’s base URL at it, and it forwards to your model server or provider. BroadNet says it inspects requests, responses, streamed chunks, streamed tool-call arguments, retrieved context and reasoning fields, and sanitises traces and logs separately.
BroadNet’s May 2026 evaluation on its own corpora reports this recall:
| Detector stack | Credential replay | Clinical PHI | Broad English PII |
|---|---|---|---|
| Regex only | 0.944 | 0.019 | 0.208 |
| Microsoft Presidio | 0.194 | 0.056 | 0.326 |
| LSDF broad-PII detectors | 1.000 | 0.944 | 0.685 |
These are a vendor’s numbers for its own tool, on corpora it chose. They are useful as a starting point, and the repository ships the replay so you can check them. LSDF works in front of any model, including ones BroadNet does not host.
What are the limits?
The paper is a preprint, not peer-reviewed. Under BroadNet’s responsible publication policy, exact prompts, credential formats and thresholds are withheld from the public version and shared with model providers and researchers on request. Model versions change fast, so a verdict from April 2026 is a reason to test, not a substitute for testing.
