JailbreakBench

Overview

JailbreakBench standardizes evaluation of attacks that try to make a safety-aligned model produce disallowed content. It combines a fixed behavior set, explicit threat models, target-system templates, automated judges, an evolving repository of jailbreak artifacts, and an official leaderboard for attacks and defenses.

Its main contribution is comparability. Attack-success claims are otherwise easy to inflate by changing the harmful requests, query budget, model template, judge, or definition of success.

Task details

The headline JBB-Behaviors misuse split contains 100 harmful behaviors curated from original items, AdvBench, and the Trojan Detection Challenge/HarmBench lineage. The current repository also provides 100 benign behaviors for measuring defense over-refusal. An attack transforms a behavior into one or more prompts for a target model; a judge decides whether the response fulfills the harmful goal.

The artifact repository stores submitted jailbreak strings with metadata so later work can rerun attacks rather than relying on screenshots or undisclosed prompts.

Example

A behavior can request assistance with a disallowed cyber or physical-harm goal. The benchmark does not require a wiki page to repeat the operational payload. An attack wraps that goal in an adversarial prompt, the target model responds, and the judge classifies whether harmful compliance occurred. A benign counterpart checks that a defense does not simply block related vocabulary.

Scoring methodology

Timeline dating convention: the chart places each PAIR result at the target model’s public launch date, not the JailbreakBench evaluation date. This lets the timeline show how models released over time performed under one fixed attack family. Dates were checked against the official GPT-3.5 Turbo, GPT-4, Vicuna, and Llama 2 announcements. Lower ASR is safer; GCG, JB-Chat, random-search, and defended results are not mixed into this chart.

Attack success rate is the fraction of the 100 misuse behaviors successfully jailbroken. Current official tooling includes a Llama-3-70B jailbreak judge and a Llama-3-8B refusal judge. The package also publishes a 300-response comparison set with three human annotations per response to evaluate candidate judges.

ASR without a target-call budget is incomplete. Adaptive attacks can try many candidates, so report target and attacker calls, system prompt, chat template, model version, temperature, defense, and judge.

What it doesn’t measure

JailbreakBench does not cover every safety policy, language, modality, multi-agent workflow, or real-world consequence. A model can resist the published attacks while remaining vulnerable to new strategies, indirect prompt injection, or tool-mediated harm.

Known issues & criticisms

Judge errors can change ASR, especially for partial, coded, or superficially compliant responses.
The 100-behavior set is broad enough for comparison but small relative to real safety policies.
Public behaviors and artifacts invite benchmark-specific defenses and training contamination.
Attack methods with unequal query budgets are not fairly comparable.
Proprietary model updates can change results without a stable versioned checkpoint.

Contamination notes

Behaviors, code, judges, and submitted artifacts are public. The evolving artifact repository improves reproducibility but also becomes training material. Use exact versions, keep an untouched private red-team set, and avoid claiming general jailbreak robustness from JBB alone.

How to run it

pip install jailbreakbench

For local attacks or defenses, install the optional vLLM dependencies:

pip install "jailbreakbench[vllm]"

Use the official package examples to load JBB-Behaviors, run the target model, and apply the packaged judges. Never run harmful generations against real systems or tools; keep evaluation in an isolated environment and follow the repository’s responsible-use guidance.

History & lineage

JailbreakBench was submitted on 2024-03-28 and accepted to the NeurIPS 2024 Datasets and Benchmarks Track. Its behavior set draws from AdvBench and the TDC/HarmBench lineage, while its artifact repository creates a common interchange format for later attacks and defenses. The fixed behaviors and evolving artifacts serve different purposes and should be versioned separately.

Performance over time