IFEval
~500 prompts, each carrying an explicit, machine-checkable constraint (format, content, or style) that a response must respect.
92 AI benchmarks with their saturation status, contamination risk and the source of every score. Filter the list, or scan the trust matrix below to see which capabilities still have benchmarks that separate frontier models.
~500 prompts, each carrying an explicit, machine-checkable constraint (format, content, or style) that a response must respect.
A broad code-intelligence suite spanning understanding, retrieval, completion, translation, repair, generation, and summarization.
Current 198-task offline patch benchmark derived from paid Upwork work, with historical mixed implementation and manager-task variants.
A 4,428-problem Olympiad mathematics suite spanning more than 33 subdomains and ten annotated difficulty levels.
Python code-synthesis benchmark that rewards both functional correctness and runtime efficiency against real solution distributions.
Meta's evolving suite for insecure code, cyberattack assistance, prompt injection, exploitation, SOC analysis, and autopatching.
Real GitHub issues from 12 Python repositories, solved by generating patches that must pass repository tests.
GSM8K test problems with unusually large numerical values, designed to separate reasoning from easy arithmetic.
Parallel HumanEval and MBPP translations for comparing code generation across many programming languages.
204-task community-built mega-suite spanning reasoning, knowledge, math, code and social bias — the sprawling proving ground BBH was distilled from.
DeepMind's competitive-programming corpus with temporally split problems, human submissions, and generated correctness tests.
Every benchmark by capability and status. Click a benchmark for its evidence and scores.