Have Any LLMs Reliably Resisted Prompt Injection?

Getting your Trinity Audio player ready...

Short answer: No—not in a security-relevant sense. Some experiments have reported prompt-injection attack success rates below 1% on particular benchmarks, and sometimes 0% against selected attacks. But these results do not show that an LLM can reliably resist repeated, adaptive attacks. A small failure rate per attempt can become a large cumulative risk when an attacker can try again.

Prompt injection occurs when a large language model treats untrusted content—such as text from a webpage, email, document, database or tool response—as an instruction that competes with the system’s intended instructions. The problem is especially serious for AI agents that can access confidential information or perform external actions.

This limitation has practical consequences for AI Internet-Meritocracy (AIIM), which uses AI to evaluate public scientific and open-source contributions. AIIM does not rely exclusively on another LLM to decide whether a user deliberately manipulated the evaluator. Suspected serious abuse can instead lead to human ban/unban voting and an appeal process.

Why a 1% attack success rate is not secure

Suppose each attack has an independent probability (p) of succeeding and the attacker makes (n) attempts. The probability that at least one attack succeeds is:

P(at least one success) = 1-(1-p)^n.
]

If the success probability is 1% per attempt, 100 independent attempts produce a cumulative success probability of:

1-0.99^{100}\approx 63.4%.

Thus, a 1% per-attempt success rate does not mean that the system is 99% secure. Under this simplified model, an attacker has better-than-even odds after about 69 attempts and a greater than 99% chance after about 459 attempts.

Attack success per attempt	Probability of at least one success in 100 attempts
0.1%	9.5%
1%	63.4%
2%	86.7%
8%	99.98%

These calculations are illustrative, not predictions of a particular deployment. Real attempts may not be independent, and rate limits, account blocking and human approval can reduce the number of opportunities. On the other hand, a real attacker can learn from failures and adapt subsequent prompts, so assuming an unchanged failure probability may be too optimistic.

The security-relevant question is therefore not merely, “What percentage of a fixed test set did the defense stop?” It is:

Can the system remain safe across repeated attempts by an attacker who knows the defense, observes its responses and changes strategy?

No cited experiment establishes that general-purpose LLMs meet that standard.

Experiments that initially appear to show strong resistance

Several defenses have produced impressive benchmark results. They demonstrate meaningful risk reduction, but not reliable immunity.

Defense or evaluation	Strong reported result	Why it is not conclusive
Spotlighting	Attack success reduced from over 50% to below 2%	Tested particular transformations, models and attack sets
StruQ	Below 2% against tested manual attacks	An optimization-based GCG attack still achieved 58% on Llama
SecAlign	0% against tested optimization-free attacks	Stronger optimization-based attacks succeeded mainly at rates below 10%, not 0%
PromptArmor	Below 1% on AgentDojo	A detector-and-filter pipeline on a benchmark, not proof against indefinite adaptive attack
Adaptive Attacks study	None of eight defenses held	Adaptive attacks exceeded 50% success against every tested defense
Adaptive Adversaries benchmark	0–1% on the first attack turn	Success rose to 5.4–14.0% when attackers received 15 adaptive rounds

Spotlighting reduced attacks below 2%

Microsoft researchers proposed Spotlighting, which transforms or marks untrusted text to help an LLM distinguish its source. In their experiments with GPT-family models, attack success fell from above 50% to below 2%, with little effect on ordinary task performance.

This is evidence that provenance signals help. It is not evidence that the model will continue to resist an attacker who can make hundreds of attempts or design attacks for the known transformation.

Source: Hines et al., “Defending Against Indirect Prompt Injection Attacks With Spotlighting”, 2024.

StruQ resisted manual attacks but not a stronger optimized attack

StruQ separates trusted instructions from untrusted data and fine-tunes the model to obey only the instruction channel. The peer-reviewed USENIX Security paper reported attack success below 2% across tested manual attacks.

However, the same paper reported a 9% success rate for Tree of Attacks with Pruning and a 58% success rate for the optimization-based Greedy Coordinate Gradient attack on Llama. The authors explicitly stated that StruQ was not fully secure.

Source: Chen et al., “StruQ: Defending Against Prompt Injection with Structured Queries”, USENIX Security 2025.

SecAlign reached 0% only under a restricted set of attacks

SecAlign uses preference optimization to train a model to prefer responses to legitimate instructions over responses requested by injected instructions. Its researchers reported 0% success for the tested optimization-free attacks while maintaining general utility.

Against stronger optimization-based attacks, however, success rates were mainly below 10% rather than zero. That is a substantial improvement over earlier defenses, but it is still unsafe if consequential attacks can be repeated.

Source: Chen et al., “SecAlign: Defending Against Prompt Injection with Preference Optimization”, ACM CCS 2025.

PromptArmor reported below 1% on AgentDojo

PromptArmor uses a separate LLM to identify and remove suspected injected instructions before the protected agent processes the input. Its authors reported false-positive and false-negative rates below 1% with several guardrail models on AgentDojo, and an attack success rate below 1% after filtering. They also evaluated attacks adapted to the defense.

This is a strong benchmark result. Yet “below 1% on AgentDojo under the tested adaptive strategies” is narrower than “secure under sustained attack.” A low residual rate can still accumulate, while future attackers may search outside the tested distribution.

Source: Shi et al., “PromptArmor: Simple yet Effective Prompt Injection Defenses”, 2025.

Adaptive experiments reverse the conclusion

The most important evidence comes from experiments that allow attackers to target the defense rather than replay a fixed collection of prompts.

Zhan and colleagues evaluated eight defenses against indirect prompt injection and developed adaptive attacks for each. The attacks bypassed all eight defenses, consistently achieving success rates above 50%. This does not prove that every possible defense must fail, but it shows why low success rates against non-adaptive test sets can be misleading.

Source: Zhan et al., “Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents”, Findings of NAACL 2025.

A July 2026 preprint provides an especially direct test of repeated attack. In 21 scenarios, restricting evaluation to the attacker’s first turn produced only 0–1% attack success. Giving an autonomous attacker 15 rounds in which it could observe responses and adapt raised success to 5.4–14.0%, depending on the defender. Because this is a recent preprint, its findings should be treated as provisional, but the comparison isolates a crucial methodological problem: a single-turn score can substantially understate multi-round risk.

Source: Jain, Hartmann and Li, “Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security”, 2026 preprint.

Did the LLM resist, or did the system prevent the action?

Model resistance and system security are not the same achievement.

In AgentDojo, a simple tool-filtering defense reduced attack success to 7.5%. Its protection came from limiting the tools available for the current task. For example, an agent that only needs to read information may not be given a tool that can send or modify it. The paper also explains the limitation: filtering can fail when legitimate and malicious objectives require the same tools or when the needed tools cannot be predicted in advance.

Source: Debenedetti et al., “AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents”, NeurIPS 2024.

This suggests a more defensible design principle:

Do not assume that an LLM will always reject hostile instructions. Limit what a successful injection would be able to do.

Least-privilege permissions, deterministic authorization, separation of untrusted data, transaction limits and human approval for consequential actions can reduce harm even when the model makes a mistake. These controls do not prove that the LLM itself resisted prompt injection; they make the wider application less dependent on perfect model behavior.

Why AIIM uses human voting against suspected manipulation

AIIM separates two different jobs. The LLM evaluates evidence of scientific, software and science-marketing contributions. When there is evidence that a user deliberately tried to manipulate that evaluation—through prompt injection or another serious abuse—humans can decide whether to ban the user. Verified users can inspect relevant account information and AI audit records, initiate a ban vote with an explanation, and later initiate an unban vote. The unban procedure provides an appeal against mistakes or disproportionate sanctions.

Human voting is not evidence that people are infallible or universally better decision-makers than AI. It is an attempt to avoid asking another highly similar LLM to serve as the final judge of an attack that may exploit weaknesses shared by both models.

Victor Porton’s cognitive-independence conjecture

AIIM’s creator, Victor Porton, proposes the following explanation for why a group of people may perform better than a group of LLMs in this role:

Similarity is cheap; independence is expensive.

Current LLMs are trained on substantially overlapping Internet data, use related model architectures and are optimized through broadly similar processes. Consequently, apparently separate AI evaluators and AI appeal judges may make correlated errors. An injection that fools the first model may also fool a second model that receives the same hostile material or shares the same learned vulnerabilities.

People, by contrast, develop through different families, cultures, educations, professions and individual experiences. Porton conjectures that this cognitive diversity makes their errors less correlated: a malicious instruction capable of controlling one LLM is unlikely to control an entire diverse electorate in the same mechanical way. Under this hypothesis, the advantage of human voting is not that every voter is wiser than an LLM. It is that the voters are more genuinely independent of both the attacked model and one another.

This remains a conjecture, not an experimentally established general law. Human voters are vulnerable to persuasion, groupthink, factional bias, bribery, fatigue, strategic accusations and Sybil attacks. They may also misunderstand technical evidence. Human voting therefore needs transparent reasons, identity protections, conflict-of-interest rules, auditability and an effective appeal process. Voters should receive the evidence needed to judge the alleged conduct without unnecessarily reproducing hostile instructions throughout the system.

The AIIM architecture can be summarized as a separation of powers:

Function	Primary mechanism
Evaluate public contributions	LLM-based analysis
Enforce transaction and permission limits	Deterministic software rules
Identify suspicious patterns	AI and software monitoring
Decide intentional serious misconduct	Human ban voting
Correct sanctions	Human unban voting and appeal

Whether this hybrid arrangement actually outperforms AI-only adjudication is a testable question. Relevant measurements include prompt-injection losses, false bans, missed abuse, agreement among voters, reversals on appeal and user assessments of ban/unban fairness. The case for human voting is therefore an architectural hypothesis to test, not a substitute for adversarial evidence.

What experiments have actually demonstrated?

The evidence supports five narrower conclusions:

1. Defensive training and input structuring can greatly reduce prompt-injection success on specified benchmarks.
2. A reported 0% result means that no attack in that finite evaluation succeeded; it does not establish a true failure probability of zero.
3. Repeated and adaptive attacks can turn apparently small single-turn risks into much larger cumulative risks.
4. Security controls outside the LLM may be more dependable than asking a probabilistic model to make every security decision correctly.
5. Human voting is a plausible source of independent oversight, but its proposed advantage over correlated LLM judges still requires direct empirical testing.

Therefore, it is fair to say that some LLM defenses have performed well in bounded experiments. It is not fair to conclude that any LLM has demonstrated reliable resistance to prompt injection under sustained adversarial pressure.

Prompt injection, AI alignment and superintelligence

Prompt-injection resistance is related to AI alignment because it tests whether a model continues to follow its intended objective when exposed to competing instructions. But it is only one narrow component of alignment. A system may reject prompt injections while still misunderstanding its objective, acting on false information or receiving excessive authority from its designers. AIIM’s human-voting proposal treats diverse human judgment as an external check on correlated model failures rather than assuming that an AI can reliably judge another similar AI.

The limitation becomes more serious when considering superintelligence. Today’s experiments test present systems against bounded human or model-generated attacks. They do not demonstrate that the same defenses would remain effective against a more capable adversary, nor that a superintelligent system itself would remain aligned.

Conclusion

No experiment has demonstrated that a general-purpose LLM can reliably resist prompt injection under repeated, adaptive attack. Some defenses have reduced per-test attack success below 1% and even recorded 0% on selected test sets. Those are useful improvements, not security guarantees.

A system handling money, confidential data or consequential decisions should therefore assume that prompt injection will sometimes succeed. Its safety should depend on layered controls: least privilege, deterministic authorization, isolation of untrusted content, rate limits, monitoring and human approval for high-impact actions. AIIM applies this principle by combining AI evaluation with deterministic constraints and human ban/unban voting for suspected deliberate abuse.

The scientifically defensible conclusion is:

Prompt injection can be made less likely, but current experiments do not show that LLMs can resist it reliably over repeated adaptive attempts. Diverse human voting may provide more independent oversight than an ensemble of similar LLM judges, but that conjecture must itself be tested.