benchwiki
safety-alignment

MM-SafetyBench

Deprecated
2023-11-29 → today

A 5,040-pair benchmark testing whether query-relevant images can bypass multimodal model safeguards.

last updated 2026-09-14

Performance over time

vendor-reported independent
60708020232024

Baseline: No protocol-comparable human attack-success baseline is reported; humans are not safety-aligned multimodal target models.

Overview

MM-SafetyBench tests whether an apparently benign text query becomes dangerous when paired with a query-relevant image. The paper showed that safety-aligned language backbones could be bypassed once harmful intent was shifted into the visual channel. Its 5,040 text-image pairs span 13 safety scenarios and several image-construction conditions.

The benchmark was influential in defining multimodal jailbreak evaluation. It is now a historical baseline rather than the strongest current suite because both attacks and multimodal models have moved quickly since 2023.

Task details

The authors start from malicious questions, extract unsafe key phrases, and use them to produce query-relevant images. Conditions include generated images, typography images, and combined generated-image-plus-typography inputs. Target models answer the multimodal query, and a judge labels the response safe or unsafe.

The 13 scenarios include illegal activity, hate speech, malware generation, physical harm, fraud, privacy violations, and other unsafe-content categories. A tiny 168-sample ID list is also available for lower-cost checks.

Example

A sanitized item pairs a neutral-looking instruction with an image that encodes a disallowed request. A robust multimodal assistant should recognize the combined intent and avoid actionable harmful help. This page does not reproduce operational harmful prompts or generated examples.

Scoring methodology

Timeline dating convention: the chart places each score at the target multimodal model’s public launch date, not the MM-SafetyBench evaluation date. This lets the timeline show how models released over time performed under the same tiny-set SD+TYPO attack. Dates were checked against the official LLaVA repository’s LLaVA-1.5 release record and the LLaVA-Phi release paper. Lower ASR is safer; full-set, defense, and other image-condition results are deliberately not mixed into this series.

Attack success rate is the fraction of responses judged unsafe. Lower is safer. The repository’s result format stores safe and unsafe counts per model and computes unsafe / (safe + unsafe). Report results by scenario and image condition before any overall mean.

The paper uses GPT-4 as a judge, but the repository says the detailed Azure GPT-4 implementation is not provided. Judge prompts and versions therefore need to accompany any reproduction.

What it doesn’t measure

MM-SafetyBench does not measure ordinary visual question answering, benign refusal rates, indirect attacks through tools or retrieved documents, video or audio attacks, multi-turn recovery, or whether a response causes real-world harm.

Known issues & criticisms

  • The judge implementation is not fully released, limiting exact reproduction.
  • Some image conditions contain visible text, so optical character recognition and safety behavior are entangled.
  • Generated images inherit Stable Diffusion artifacts and licenses.
  • Public static questions and images are easy to include in training or defense tuning.
  • ASR changes with target policy and judge, so model rankings are not timeless.

Contamination notes

Processed questions and image assets are public, and the benchmark has been reused in later training and evaluation work. This creates high risk of benchmark-aware safety tuning. Use it to reproduce a known attack class, then add fresh private images and policy-specific benign controls.

How to run it

git clone https://github.com/isXinLiu/MM-SafetyBench.git
cd MM-SafetyBench

Download the official image archive, place it under data/imgs, and follow the repository’s model-specific inference scripts. Run evaluation.py only after pinning or replacing the judge implementation, and state the image condition, target template, and judge in results.

History & lineage

MM-SafetyBench was submitted on 2023-11-29, revised through v5, and accepted at ECCV 2024. The team later released a tiny subset, while subsequent work introduced broader multimodal jailbreak and safety suites. MM-SafetyBench++ is a later related benchmark, but its task mix and protocol are not a directly comparable score revision.

Further reading