Benchmarks

92 AI benchmarks with their saturation status, contamination risk and the source of every score. Filter the list, or scan the trust matrix below to see which capabilities still have benchmarks that separate frontier models.

Capability Domains
Status
Contamination
Sort By
 
Displaying 11 of 11 benchmarks (filtered from 92)
Instruction FollowingDeprecated

IFEval

~500 prompts, each carrying an explicit, machine-checkable constraint (format, content, or style) that a response must respect.

Trust scoreout of 100
Low contamination
CodingDeprecated

CodeXGLUE

A broad code-intelligence suite spanning understanding, retrieval, completion, translation, repair, generation, and summarization.

Trust scoreout of 100
Medium contamination
CodingDeprecated

SWE-Lancer

Current 198-task offline patch benchmark derived from paid Upwork work, with historical mixed implementation and manager-task variants.

Trust scoreout of 100
Medium contamination
MathematicsDeprecated

Omni-MATH

A 4,428-problem Olympiad mathematics suite spanning more than 33 subdomains and ten annotated difficulty levels.

Trust scoreout of 100
High contamination
CodingDeprecated

Mercury

Python code-synthesis benchmark that rewards both functional correctness and runtime efficiency against real solution distributions.

Trust scoreout of 100
High contamination
CodingDeprecated

CyberSecEval

Meta's evolving suite for insecure code, cyberattack assistance, prompt injection, exploitation, SOC analysis, and autopatching.

Trust scoreout of 100
High contamination
CodingDeprecated

SWE-bench

Real GitHub issues from 12 Python repositories, solved by generating patches that must pass repository tests.

Trust scoreout of 100
High contamination
MathematicsDeprecated

GSM-Hard

GSM8K test problems with unusually large numerical values, designed to separate reasoning from easy arithmetic.

Trust scoreout of 100
High contamination
CodingDeprecated

MultiPL-E

Parallel HumanEval and MBPP translations for comparing code generation across many programming languages.

Trust scoreout of 100
High contamination
ReasoningDeprecated

BIG-Bench

204-task community-built mega-suite spanning reasoning, knowledge, math, code and social bias — the sprawling proving ground BBH was distilled from.

Trust scoreout of 100
High contamination
CodingDeprecated

CodeContests

DeepMind's competitive-programming corpus with temporally split problems, human submissions, and generated correctness tests.

Trust scoreout of 100
High contamination

Trust matrix

Every benchmark by capability and status. Click a benchmark for its evidence and scores.

Contamination Risk
Refresh Cadence
Showing 92 of 92 benchmarks
CapabilityActiveNearing SaturationSaturatedDeprecated
Knowledge
Reasoning
Mathematics
Coding
Instruction Following
Long Context
Safety & Alignment
Multimodal Vision
Agentic & Tool Use