Your Guardrail Is a Text Classifier

TextFooler appeared as a preprint in 2019 and was published at AAAI 2020. It ranks the words in a piece of text by how much each one contributes to a classifier’s decision, then swaps the important ones for synonyms until the classifier changes its mind while a human reads the same meaning. For years it was a benchmark technique in a corner of natural language processing research, useful mainly for measuring how brittle sentiment classifiers were.

In April 2025, a team at Mindgard pointed TextFooler and its relatives at commercial AI security products. They tested six protection systems: Azure AI Content Safety Prompt Shield, Meta’s Prompt Guard, Protect AI’s fine-tuned DeBERTa in two versions, Vijil Prompt Injection and NeMo Guard Jailbreak Detect. Some technique-and-guardrail pairs reached 100% evasion, with the injection or jailbreak payload still working on the model behind the filter. TextFooler alone reached a 46.27% attack success rate against prompt injection detection and 48.46% against jailbreak detection.

The reason is not subtle. Most prompt injection detectors are text classifiers. So are most jailbreak detectors, toxicity filters, content moderation models and PII scanners. Rules, canonicalisation and LLM-as-judge approaches exist, but the products sold as guardrails are overwhelmingly classifiers. Much of the industry took one of the most thoroughly broken model classes in the adversarial machine learning literature, put it in front of a language model, and sold it as a shield.

Here is the line for whoever approved that purchase. These products reduce a security decision to text classification, which inherits a model class with a decade of adversarial-evasion literature behind it, and the vendor’s detection rate was measured against attackers who were not adapting. Buy them to raise the cost of the trivial majority of attacks, and never as the control that makes an agent safe to give credentials to.

What a perturbation attack is, and why text is the awkward case

A perturbation attack changes an input just enough to flip a model’s decision while leaving its meaning intact for a person. This is an adversarial example, and it happens at inference time against a finished model. Nothing is poisoned and no training data is touched, which separates it cleanly from data poisoning and from backdoors.

Images made the concept famous, and they also made it easier to reason about. Pixel values are continuous, so a norm bounds the perturbation and holds it below what an eye resolves. The size of the change is then a number you can state.

Text does not work that way, and the difference matters more than the literature usually admits. There is no continuous space between “excellent” and “superb.” You cannot move a word part of the way; you swap in another one, and the smallest step available is a whole word. So the constraint that makes an image attack imperceptible has no direct equivalent, and the field substituted proxies: semantic similarity scores, grammaticality checks, and human judgement of whether the meaning survived.

Those proxies are why text perturbation attacks are practical against a deployed system. An attacker who does not care whether the text reads naturally has an enormous search space. An attacker who does care still has synonym substitution, typos, word reordering and, in the Unicode era, a large supply of characters that render one way and tokenise another.

The two families, and both work

The Mindgard study splits the attack surface into two families, and the defences for them are different.

Character injection exploits the gap between what a classifier tokenises and what the language model behind it understands. Zero-width characters, homoglyphs, Unicode tag characters and emoji smuggling all leave a prompt that the model reads correctly and the classifier does not. The study tested twelve such techniques. Some, emoji smuggling among them, fully bypassed detection across several guardrails including Azure Prompt Shield.

This is not an adversarial machine learning problem at all. It is an encoding normalisation bug, of the kind web application security solved twenty years ago and then had to solve again for every new parser. The fix is the same as it was then: normalise input to a canonical form before the classifier sees it. Reject what will not normalise.

Adversarial machine learning evasion is the harder family, and it is the one the 2022 version of this article was about. The technique computes a word-importance ranking, using gradients, word removal or saliency, and then perturbs the highest-ranked words while preserving meaning. TextFooler, BERT-Attack, BAE, Alzantot and Pruthi are the named methods, and TextAttack is the open-source library that implements them.

TextAttack appeared in the 2022 version of this article as a reference for what perturbation attacks are. In the 2025 study it is the tool used to break commercial guardrails. The research did not change. Its target did.

Read the numbers with their conditions attached

The headline is up to 100% evasion, and it is accurate for specific technique-and-guardrail pairs. The per-technique numbers underneath that headline are more informative.

TextFooler was the strongest of the algorithmic methods, at 46.27% against prompt injections and 48.46% against jailbreaks. Bert-Attack and BAE performed reasonably on prompt injections and worse on jailbreaks. Alzantot and Pruthi were weak throughout. The authors give a mechanical explanation for the prompt-injection and jailbreak gap: jailbreak prompts are longer and more complex, so perturbing individual words moves the classification less.

The access assumption is the part to carry into a threat model, and it runs the opposite way to the one most buyers assume. Five of the six are open-source classifiers the authors downloaded and ran locally; only Azure’s Prompt Shield was queried as a hosted black box. Word-importance rankings computed offline against an open guardrail carried over to Azure’s hosted one, so a vendor keeping its internals private did not stop the attack. An attacker downloads an open-source guardrail, computes rankings against it, and points the result at your hosted one.

The other finding worth quoting to a vendor: no single guardrail consistently outperformed the others across attack types. Each had significant weaknesses depending on the technique.

Where perturbation attacks still hit the classical stack

The guardrail story is the current one, and it should not eclipse the fact that the original targets are still deployed and still exposed.

Spam filtering, phishing detection, log anomaly detection, malware classification and content moderation all run text and text-adjacent classifiers in production, most of them inside security tooling. Perturbation attacks against those models are the same attacks, aimed at systems whose failure is a security failure by definition. The uncomfortable version of this is that a security team running an ML-based detection stack has the same exposure it warns its business about, and usually less instrumentation.

That connects to a pattern across the archive. Label flipping also succeeds most easily against spam filters and malware detectors, for the same reason: security tooling adopted machine learning early, and the models are old, small and rarely re-evaluated adversarially.

What to do

Normalise before you classify. Unicode normalisation, homoglyph folding, zero-width character stripping and a rejection rule for input that does not round-trip. This eliminates or substantially reduces the Unicode and parser-differential variants, which are the cheaper and more reliable half of this surface, and it is a parsing fix rather than a machine learning one. Emoji-based semantic encodings run wider than Unicode normalisation reaches.

Assume the classifier will be evaded and design for it. The OWASP 2026 revision moved Excessive Agency from sixth to third, and the reasoning applies directly here. A guardrail that fails in front of a model that can only produce text is a bad day. The same failure in front of an agent holding credentials is an incident. Bound what happens after the guardrail loses.

Test your guardrail with the attacker’s tools. TextAttack is open source. Garak and PyRIT automate batteries of known attacks. Run them against your own stack at every release and treat the result as a penetration test rather than a benchmark. If your vendor cannot tell you their detection rate under TextFooler, you have learned something.

Ask vendors for adaptive evaluation, not detection rates. A detection rate measured on unmodified attack corpora tells you how the product performs against an attacker who is not adapting. Every result in this article comes from attackers who adapted, using public tools.

Do not stack identical classifiers and call it defence in depth. Two text classifiers trained on similar data share blind spots. Independence is the property that makes layering work, and two guardrails from the same paradigm do not have it.

Where the obligations sit

Prompt Injection remains LLM01 in the OWASP Top 10 for LLM Applications 2026, published 3 August 2026, and Excessive Agency climbed from sixth to third in the same revision. Read together, those two rankings are an argument that the field has stopped expecting the input filter to hold.

Article 15(5) of the EU AI Act names adversarial examples and model evasion among the attack classes that technical solutions for high-risk systems must, where appropriate, prevent, detect, respond to, resolve and control for. That obligation applies from 2 December 2027 for standalone Annex III systems and 2 August 2028 for Annex I embedded systems, per Regulation (EU) 2026/1744, in force since 27 July 2026. Checked 6 September 2026, with the full timeline in data poisoning.

Related coverage: adversarial attacks for the general mechanism, model evasion for the detection-bypass case, and AI security fundamentals for the map.