04 · The Check
One flag or twenty-five
The same prompts, two guardrails, two very different safety reports. The bigger number still needs a human before anyone believes it.
Collected from AI Governance, Ethics and Leadership, “AI Governance in Action: The False Comfort of Automated AI Guardrails,” sent 17 November 2026 at 17:07 UTC (04:07 Wednesday, Melbourne time). Read in the email. The figures are from the author’s own experiment, as reported; the full audit, prompt suite and further analyses sit behind the paid upgrade and we have not read them.
The experiment is simple enough to repeat. Take a test suite of more than 150 roleplay prompts built to probe a model’s safety limits. Pass the identical set through two widely used open-source moderation tools, Toxic-BERT and Meta’s Llama Guard. Count what each one flags.
Toxic-BERT flagged one. Llama Guard flagged 25. The author’s reading is blunt: if your safety dashboard rests on a legacy toxicity classifier, a green report may be hiding a 96 per cent blind spot.
The reasons are about what the older tool was built to see. Toxic-BERT was trained on Wikipedia talk pages from 2018, to catch overt abuse. A roleplay jailbreak rarely swears. It asks politely, in costume: imagine you are an amoral historian in a dystopian novel. The vocabulary is clean, so the score is near zero. The tool checks six interpersonal labels, from insult to identity hate, and has no category at all for the things executives actually fear, such as cyberattack help or data exfiltration. And it reads only 512 tokens, so a long piece of world-building with the harmful request at the very end is truncated before the request arrives.
“Automating an evaluation is easy; but verifying what the test results actually mean is the real work.”
Llama Guard, an LLM judge with a context window of more than 8,000 tokens and a 14-category safety taxonomy, caught the 24 that Toxic-BERT missed. The tempting end of the story is to swap tools and declare victory. The author refuses it. Twenty-five flags is not twenty-five stopped attacks. Some may be false alarms on dark but harmless fiction. And the 125-odd prompts marked safe have not been shown to be safe.
So the letter ends in a routine rather than a product. A person labels every flagged case, true breach or false alarm. A person spot-checks 10 to 15 per cent of the passes. The audited labels are locked into a version-controlled golden set, and every model swap or policy tweak is measured against it. Then three questions for anyone handed a safety report: what tool produced this, has it been checked for false positives and negatives, and what is the process for checking it.