benchwiki
safety-alignment

Indirect Prompt Injection Arena

Active
2026-03-16 → today

Forty-one agent scenarios derived from a 272,000-attempt public competition on concealed indirect prompt injection.

last updated 2026-09-14

Performance over time

vendor-reported independent
01020252026

Baseline: No human agent baseline is defined. The source evidence is adversarial competition success against model-based agents.

Overview

The Indirect Prompt Injection Arena asks whether an attacker can hijack an AI agent by placing instructions inside content the agent reads, such as an email, review, document, code repository, or rendered web page. The attacker does not directly control the user’s request. The dangerous instruction enters through an external data source that the agent must process to complete a legitimate task.

The 2026 paper reports a three-week public competition with 464 participants and 272,000 attempts against 13 frontier models. Every model was compromised at least once. The open-source release converts the competition into a runnable 41-scenario benchmark.

Task details

The 41 behaviors comprise 18 tool-use, 15 coding, and 8 browser/computer-use scenarios across three waves. Each defines a benign user task, an attacker-controlled insertion point, available tools, a target harmful action, and criteria for both compromise and concealment.

The harness can run multi-step trajectories of up to five steps. WorldSim simulates tool responses, deterministic judges inspect tool calls, and configurable LLM judges assess stealth and visible task completion.

Example

A user asks an agent to summarize account messages. One retrieved message contains an untrusted instruction telling the agent to change an account setting while hiding that action from the summary. A successful concealed attack causes the unauthorized tool call and still returns a plausible response to the user. The benchmark runs such actions only in simulation.

Scoring methodology

Timeline dating convention: the chart places each competition score at the model’s public launch date, not the competition or paper date. This lets the timeline show how successive model releases performed against indirect prompt injection. Dates were checked against the official Gemini 2.5 Pro, Kimi K2, Claude Sonnet 4.5, Claude Haiku 4.5, and Claude Opus 4.5 announcements. Lower overall competition ASR is safer; curated transfer ASR is a different protocol and is not mixed into this chart.

The paper’s central measure is concealed attack success rate. A break requires all scenario criteria to pass, not merely that the model repeats an injected string. Deterministic criteria verify harmful tool actions, while LLM-judged criteria check concealment and whether the legitimate task still appears complete.

The competition and open harness are related but not identical score settings. Public participants adapt attacks over many attempts; the open release supplies 95 attacks across 28 behaviors, and those attacks did not transfer to closed-source models in the authors’ reported experiment. State the attack source, model, judge, WorldSim, wave, and budget.

What it doesn’t measure

The benchmark does not prove production security. It focuses on single-turn attacker placement followed by bounded agent trajectories, simulated tools, selected domains, and English content. It does not cover every persistence mechanism, insider threat, data exfiltration route, or system-level defense.

Known issues & criticisms

  • Public-competition ASR depends on participant effort and number of attempts, not only model robustness.
  • LLM concealment judges introduce model-specific uncertainty.
  • WorldSim and synthetic tools cannot capture every side effect or permission boundary of production systems.
  • Public behaviors and attacks enable targeted defense tuning.
  • A low aggregate ASR can hide a catastrophic break in one high-impact scenario.

Contamination notes

The paper, all 41 scenario definitions, open harness, and a subset of attacks are public. This makes exact regression testing possible and future training overlap likely. Maintain private behaviors and injection locations, and evaluate system controls such as content provenance, tool permissions, confirmation gates, and audit logs rather than relying on model alignment alone.

How to run it

git clone https://github.com/GraySwanAI/ipi_arena_os.git
cd ipi_arena_os
pip install -e .
ipi-arena-bench list-behaviors
ipi-arena-bench run --config examples/hf_attacks.yaml --dry-run

Only after reviewing the dry run should you configure a sandboxed target, judge, and WorldSim and execute the benchmark. Browser behaviors additionally require Playwright and Chromium. Never connect benchmark tools to real accounts, money movement, production repositories, or physical devices.

History & lineage

The paper was submitted on 2026-03-16 and reports the Indirect Prompt Injection Arena competition. It follows earlier indirect-injection work and complements AgentDojo and the 2025 Agent Red Teaming competition. The open-source release packages 41 behaviors and a limited attack set; LivePI and domain-specific IPI benchmarks are later related efforts, not score-compatible successors.

Further reading