benchwiki
safety-alignment

SafetyBench

Nearing saturation
2023-09-13 → today

A bilingual 11,435-question multiple-choice test of safety knowledge across seven risk categories.

last updated 2026-09-14

Performance over time

vendor-reported independent
6575859520222023

Baseline: No single protocol-comparable human aggregate is published on the official benchmark page. Questions have answer keys, but that is not a measured human baseline.

Overview

SafetyBench evaluates whether a language model can recognize safer answers to multiple-choice questions. Its 11,435 items span seven categories and are provided in Chinese and English. Because scoring is exact-match, it is much cheaper and more reproducible than open-ended safety judging.

That efficiency comes with a sharp limit: recognizing a safe choice is not the same as refusing a harmful request or behaving safely while using tools. SafetyBench is a safety-knowledge exam, not a deployment red-team.

Task details

Questions cover offensiveness, unfairness and bias, physical health, mental health, illegal activities, ethics and morality, and privacy and property. They include knowledge questions, comparisons, dialogue scenarios, and everyday situations. Each has one correct option.

The release includes Chinese and English test inputs plus a Chinese subset with highly sensitive keywords removed for APIs with strict input filtering. Five development examples per category and language support few-shot evaluation. Official test labels are withheld.

Example

A sanitized item may describe an online-account scenario and ask which action best protects another person’s privacy. The model returns the option letter. The evaluator checks that letter against the hidden answer key; it does not grade an explanation.

Scoring methodology

Timeline dating convention: the chart places each benchmark score at the model’s public launch date, not when the SafetyBench experiment was executed. This lets the timeline show how successive model releases performed on SafetyBench. Dates were checked against the Flan-T5 release paper, OpenAI’s GPT-3.5 Turbo and GPT-4 announcements, and the official ChatGLM2 release record in the THUDM repository. Only English zero-shot average accuracy is plotted.

The primary metric is accuracy. Zero-shot and five-shot runs are separate, as are the three language/subset tracks. The official workflow formats model outputs, extracts a choice, and submits predictions for hidden-label scoring.

The paper also sampled open-ended reformulations and reported correlations with multiple-choice accuracy. Those analyses support construct validity but do not make generated-response safety interchangeable with MCQ accuracy.

What it doesn’t measure

SafetyBench does not directly test jailbreak resistance, refusal calibration, harmful completion severity, long conversations, multimodal attacks, tool use, or policy compliance under adversarial pressure. It also excludes politically sensitive topics whose correct answers may differ across contexts.

Known issues & criticisms

  • Some data were translated between Chinese and English, so artifacts can change difficulty and cultural meaning.
  • ChatGPT helped augment questions; the paper reports a measurable advantage on augmented data, even though its aggregate conclusion was limited.
  • Multiple-choice options expose the intended safety distinction and reward recognition or guessing.
  • The broad aggregate can hide weak performance in a high-impact category.
  • Public inputs and source-dataset reuse create contamination risk even while labels remain hidden.

Contamination notes

The official test questions are downloadable, while answer labels are held by the submission server. This is better than publishing both sides but does not prevent input memorization, source-data overlap, or later leakage from result artifacts. Treat post-release training as a medium contamination risk and disclose whether any SafetyBench-derived data were used.

How to run it

git clone https://github.com/thu-coai/SafetyBench.git
cd SafetyBench

Or load the public inputs:

from datasets import load_dataset

test = load_dataset("thu-coai/SafetyBench", "test")
dev = load_dataset("thu-coai/SafetyBench", "dev")

Generate one option per item, follow the official output format, and submit predictions through the official workflow. Record language track and shot count.

History & lineage

SafetyBench was submitted on 2023-09-13. It consolidated seven safety-knowledge categories into a bilingual MCQ format and added a filtered Chinese subset for restrictive APIs. Later safety benchmarks increasingly emphasize open-ended generation, refusal calibration, moderation, and agent behavior; those complement rather than directly replace its knowledge axis.

Further reading