Benchmarks

92 AI benchmarks with their saturation status, contamination risk and the source of every score. Filter the list, or scan the trust matrix below to see which capabilities still have benchmarks that separate frontier models.

Capability Domains
Status
Contamination
Sort By
 
Displaying 20 of 20 benchmarks (filtered from 92)
Long ContextSaturated

Needle-in-a-Haystack

Synthetic retrieval stress test that hides facts at different depths and context lengths, then asks the model to recover them.

Trust scoreout of 100
Low contamination
CodingSaturated

LiveCodeBench

Continuously updated contest problems for time-windowed code generation, execution, test prediction, and self-repair evaluation.

Trust scoreout of 100
Medium contamination
ReasoningSaturated

WinoGrande

Binary fill-in-the-blank commonsense problems, scaled from Winograd schemas and adversarially filtered to reduce dataset shortcuts.

Trust scoreout of 100
High contamination
ReasoningSaturated

HellaSwag

Choose the plausible continuation of an everyday event from one real ending and three adversarially filtered machine generations.

Trust scoreout of 100
High contamination
ReasoningSaturated

CommonsenseQA

Five-choice questions derived from ConceptNet relations, designed to require everyday knowledge beyond a supplied passage.

Trust scoreout of 100
High contamination
ReasoningSaturated

GLUE

Nine-task English NLU suite combining acceptability, sentiment, similarity, paraphrase, inference and coreference into one score.

Trust scoreout of 100
High contamination
CodingSaturated

SWE-bench Verified

A 500-task, engineer-reviewed subset of SWE-bench that became the standard repository-level coding-agent test.

Trust scoreout of 100
High contamination
KnowledgeSaturated

MMLU-Redux

A manual error audit of MMLU: experts re-annotated thousands of questions and found ~6.5% are broken — the evidence base for MMLU's label-noise ceiling.

Trust scoreout of 100
High contamination
CodingSaturated

HumanEval+

HumanEval's 164 Python tasks re-evaluated with roughly 80 times more tests to expose plausible but incorrect programs.

Trust scoreout of 100
High contamination
ReasoningSaturated

AGIEval

Standardized human exams — SATs, LSATs, China's Gaokao and civil-service tests — repurposed to grade models against real human test-takers.

Trust scoreout of 100
High contamination
ReasoningSaturated

BIG-Bench Hard

23 BIG-Bench tasks where models trailed humans — the benchmark that proved chain-of-thought prompting works, then fell to the reasoning-model era.

Trust scoreout of 100
High contamination
MathematicsSaturated

TabMWP

Grade-level math word problems that require combining a question with structured, textual, or image-rendered tables.

Trust scoreout of 100
High contamination
MathematicsSaturated

GSM8K

8.5K grade-school math word problems (2–8 arithmetic steps) — the default elementary-math benchmark and the classic chain-of-thought demonstration.

Trust scoreout of 100
High contamination
Safety & AlignmentSaturated

TruthfulQA

817 adversarial questions probing whether models repeat common human misconceptions — famous for finding that bigger models were often less truthful.

Trust scoreout of 100
High contamination
CodingSaturated

MBPP

Mostly Basic Programming Problems: 974 crowd-sourced Python synthesis tasks intended for entry-level programmers.

Trust scoreout of 100
High contamination
CodingSaturated

HumanEval

OpenAI's 164 hand-written Python function-synthesis tasks, graded by executing generated completions against unit tests.

Trust scoreout of 100
High contamination
MathematicsSaturated

MATH

12,500 AMC/AIME-level competition problems with worked solutions — usually evaluated on its 500-problem MATH-500 subset, the reasoning-model math standard.

Trust scoreout of 100
High contamination
KnowledgeSaturated

MMLU

57-subject multiple-choice exam spanning STEM, humanities, and social sciences — the defining knowledge benchmark of the 2020–2024 era.

Trust scoreout of 100
High contamination
ReasoningSaturated

ARC-AGI

Colored-grid abstraction tasks that test few-shot rule induction, not memorized facts, language fluency, or school math.

Trust scoreout of 100
High contamination
ReasoningSaturated

SuperGLUE

Eight English NLU tasks spanning question answering, inference, causal reasoning, word sense, and coreference.

Trust scoreout of 100
High contamination

Trust matrix

Every benchmark by capability and status. Click a benchmark for its evidence and scores.

Contamination Risk
Refresh Cadence
Showing 92 of 92 benchmarks
CapabilityActiveNearing SaturationSaturatedDeprecated
Knowledge
Reasoning
Mathematics
Coding
Instruction Following
Long Context
Safety & Alignment
Multimodal Vision
Agentic & Tool Use