FrontierMath
Original, unpublished research-level math problems by expert mathematicians — mostly private to resist contamination. Frontier models solved under 2% at launch.
The strongest active case on the wiki. At launch (Nov 2024) state-of-the-art models solved UNDER 2% — a benchmark frontier systems almost entirely fail. Reasoning models with Python tools have since climbed (OpenAI announced o3 at 25.2% in Dec 2024, tools-on), but that figure is contested — Epoch's own independent reproductions of comparable models came in far lower (~11%). By the June 2026 v2 release Epoch added a harder Tier 4 precisely because Tiers 1–3 progress had become significant, yet the hardest tier remains far from solved. Current figures move fast and are tool/compute-dependent — consult Epoch's live dashboard; this page does not restate unverified third-party numbers. Saturation here would mean research-level, closed-form mathematics is largely automated.
Performance Timeline
Longitudinal progression of model scores against human baselines.Performance & Historical Trajectory
Empirical score progression across model release dates and evaluation rounds.
| Model | Score | Date | Source Type | Provenance |
|---|---|---|---|---|
| GLM 5.3 Flash | 25.3% | 2026-09-12 | independent | Source ↗ |
| Hy4 | 24.9% | 2026-09-12 | independent | Source ↗ |
| MiMo-V2.6-Flash | 23.1% | 2026-09-12 | independent | Source ↗ |
| Claude Opus 5.5 | 22% | 2026-09-12 | independent | Source ↗ |
| Qwen 3.8 Max Prime | 20.6% | 2026-09-12 | independent | Source ↗ |
| Nemotron 3.5 Super VL | 21.1% | 2026-09-12 | independent | Source ↗ |
| Gemini 4 Argon | 23% | 2026-09-12 | independent | Source ↗ |
| Claude Haiku 5.5 | 21.1% | 2026-09-12 | independent | Source ↗ |
| Nano Banana 2.1 | 23% | 2026-09-12 | independent | Source ↗ |
| Mistral Large 4 | 22.1% | 2026-09-12 | independent | Source ↗ |
| GPT-6.1 Sol Pro | 23% | 2026-09-12 | independent | Source ↗ |
| GPT-6.1 Sol | 23% | 2026-09-12 | independent | Source ↗ |
| Claude Sonnet 5.5 | 23% | 2026-09-12 | independent | Source ↗ |
| Z.ai: GLM 5.3 Prime | 23% | 2026-09-12 | independent | Source ↗ |
| GPT-6 Luna Pro | 21.1% | 2026-09-12 | independent | Source ↗ |
| GPT-6 Luna | 21.1% | 2026-09-12 | independent | Source ↗ |
| GPT-6 Sol Pro | 23% | 2026-09-12 | independent | Source ↗ |
| GPT-6 Sol | 23% | 2026-09-12 | independent | Source ↗ |
| Claude Opus 5.5 | 23.8% | 2026-09-12 | independent | Source ↗ |
| Xiaomi: MiMo-V2.6-Pro-UltraSpeed | 23% | 2026-09-12 | independent | Source ↗ |
| Xiaomi: MiMo-V2.6-Flash | 21.1% | 2026-09-12 | independent | Source ↗ |
| Xiaomi: MiMo-V2.6-Pro | 21.1% | 2026-09-12 | independent | Source ↗ |
| Z.ai: GLM 5.3 FlashX | 22.1% | 2026-09-12 | independent | Source ↗ |
| DeepSeek V4.1 Flash | 21.1% | 2026-09-12 | independent | Source ↗ |
| GPT-6 Astra | 23.8% | 2026-09-12 | independent | Source ↗ |
| GPT-6 Astra Pro | 23.8% | 2026-09-12 | independent | Source ↗ |
| Gemini 3.8 Flash | 23% | 2026-09-12 | independent | Source ↗ |
| Claude Fable 5.1 | 23.8% | 2026-09-12 | independent | Source ↗ |
| Hy4 | 22.1% | 2026-09-12 | independent | Source ↗ |
| Z.ai: GLM 5.3 Flash | 21.1% | 2026-09-12 | independent | Source ↗ |
| DeepSeek V4 Flash Vision Exp | 21.1% | 2026-09-12 | independent | Source ↗ |
| Claude Sonnet 5.5 | 26.2% | 2026-09-12 | independent | Source ↗ |
| Claude Haiku 5.5 | 24.2% | 2026-09-12 | independent | Source ↗ |
| GPT-6.1 Sol Pro | 26.7% | 2026-09-12 | independent | Source ↗ |
| GPT-6 Luna | 25.9% | 2026-09-12 | independent | Source ↗ |
| Gemini 4 Argon | 26.6% | 2026-09-12 | independent | Source ↗ |
| Gemini 3.8 Flash | 26.3% | 2026-09-12 | independent | Source ↗ |
| Qwen 3.8 Max Prime | 26% | 2026-09-12 | independent | Source ↗ |
| DeepSeek V4.1 Flash 1017 | 21.1% | 2026-09-12 | independent | Source ↗ |
| GPT-6.1 Terra Pro | 23% | 2026-09-12 | independent | Source ↗ |
| GPT-6.1 Terra | 23% | 2026-09-12 | independent | Source ↗ |
| MoonshotAI: Kimi K3.1 Pro | 23% | 2026-09-12 | independent | Source ↗ |
| MiniMax: MiniMax M3.1 Flash | 21.1% | 2026-09-12 | independent | Source ↗ |
| Muse Spark Open | 23% | 2026-09-12 | independent | Source ↗ |
| LiquidAI: LFM2.5-24B-A2B | 21.1% | 2026-09-12 | independent | Source ↗ |
| Muse Wonder | 23.8% | 2026-09-12 | independent | Source ↗ |
| Qwen: Qwen3.8 Max Prime | 23.8% | 2026-09-12 | independent | Source ↗ |
| Upstage: Solar Mini 4 | 20% | 2026-09-12 | independent | Source ↗ |
| SpaceXAI: Grok 4.7 | 23% | 2026-09-12 | independent | Source ↗ |
| Qwen: Qwen3.8 Omni Flash | 21.1% | 2026-09-12 | independent | Source ↗ |
| Qwen: Qwen3.8 Max (0902) | 23% | 2026-09-12 | independent | Source ↗ |
| Muse Spark 1.3 Contributor | 20% | 2026-09-12 | independent | Source ↗ |
| Muse Spark 1.3 | 23% | 2026-09-12 | independent | Source ↗ |
| Qwen: Qwen3.8 Flash | 21.1% | 2026-09-12 | independent | Source ↗ |
| DeepSeek V4-Pro | 25.6% | 2026-09-10 | independent | Source ↗ |
| DeepSeek V4-Flash | 24.1% | 2026-09-10 | independent | Source ↗ |
| Z.ai: GLM 5.3 | 23% | 2026-09-10 | independent | Source ↗ |
| Claude Fable 5.1 | 26% | 2026-09-09 | independent | Source ↗ |
| Qwen: Qwen3.8 27B | 22.1% | 2026-09-08 | independent | Source ↗ |
| Gemini 3.7 Flash | 26.2% | 2026-09-07 | independent | Source ↗ |
| Gemini 3.7 Flash | 23% | 2026-09-07 | independent | Source ↗ |
| DeepSeek V4 Pro 0813 | 23% | 2026-09-07 | independent | Source ↗ |
| Grok 4.6 | 25.8% | 2026-09-06 | independent | Source ↗ |
| Qwen: Qwen3.8 2.4T A95B | 23% | 2026-09-06 | independent | Source ↗ |
| LiquidAI: LFM2.5-2.6B | 21.1% | 2026-09-05 | independent | Source ↗ |
| ByteDance Seed: Seed 2.1 Turbo | 22.1% | 2026-09-04 | independent | Source ↗ |
| SpaceXAI: Grok 4.6 | 23% | 2026-09-04 | independent | Source ↗ |
| Muse Glimmer 30B | 22.1% | 2026-09-04 | independent | Source ↗ |
| GPT-5 | 26.3% | 2026-09-01 | independent | Source ↗ |
| Nemotron 3.5 Lightning | 20% | 2026-09-01 | independent | Source ↗ |
| GPT-5 Image Mini | 22.1% | 2026-09-01 | independent | Source ↗ |
| GPT-5 Image | 23% | 2026-09-01 | independent | Source ↗ |
| Muse Spark 1.2 Contributor | 20% | 2026-08-30 | independent | Source ↗ |
| Muse Spark 1.2 | 23% | 2026-08-30 | independent | Source ↗ |
| DeepSeek V4.1 Flash | 26% | 2026-08-26 | independent | Source ↗ |
| Space Bunny Alpha | 24.6% | 2026-08-26 | independent | Source ↗ |
| DeepSeek V4 Flash 0731 | 23.9% | 2026-08-25 | independent | Source ↗ |
| DeepSeek V4 Flash 0731 | 22.1% | 2026-08-25 | independent | Source ↗ |
| ByteDance Seed: Seed-2.0-Code | 22.1% | 2026-08-24 | independent | Source ↗ |
| GPT-6 Astra | 26.5% | 2026-08-22 | independent | Source ↗ |
| Qwen: Qwen3.7 Flash | 20% | 2026-08-21 | independent | Source ↗ |
| Kimi K3 | 23.6% | 2026-08-20 | independent | Source ↗ |
| Claude Opus 5 | 26.8% | 2026-08-18 | independent | Source ↗ |
| Claude Sonnet 5 | 25.6% | 2026-08-18 | independent | Source ↗ |
| Claude Opus 5 | 23.8% | 2026-08-17 | independent | Source ↗ |
| Gemini 3.6 Flash | 25.5% | 2026-08-15 | independent | Source ↗ |
| Gemini 3.5 Flash-Lite | 23.9% | 2026-08-15 | independent | Source ↗ |
| Gemini 3.6 Flash | 23% | 2026-08-15 | independent | Source ↗ |
| Gemini 3.5 Flash Lite | 22.1% | 2026-08-15 | independent | Source ↗ |
| MoonshotAI: Kimi K3 | 23.8% | 2026-08-09 | independent | Source ↗ |
| GPT-5.6 Sol | 26.9% | 2026-08-03 | independent | Source ↗ |
| GPT-5.5 Thinking | 26.9% | 2026-08-03 | independent | Source ↗ |
| GPT-5.6 Luna | 24.2% | 2026-08-03 | independent | Source ↗ |
| GPT-5.6 Luna Pro | 22.1% | 2026-08-03 | independent | Source ↗ |
| GPT-5.6 Luna | 22.1% | 2026-08-03 | independent | Source ↗ |
| GPT-5.6 Terra Pro | 23.8% | 2026-08-03 | independent | Source ↗ |
| GPT-5.6 Terra | 23.8% | 2026-08-03 | independent | Source ↗ |
| GPT-5.6 Sol Pro | 23.8% | 2026-08-03 | independent | Source ↗ |
| GPT-5.6 Sol | 23.8% | 2026-08-03 | independent | Source ↗ |
| Muse Spark 1.1 | 23% | 2026-08-03 | independent | Source ↗ |
| SpaceXAI: Grok 4.5 | 23% | 2026-08-02 | independent | Source ↗ |
| Hy3 | 21.1% | 2026-07-31 | independent | Source ↗ |
| Claude Sonnet 5 | 23% | 2026-07-25 | independent | Source ↗ |
| Gemini 3.1 Flash Lite Image | 22.1% | 2026-07-25 | independent | Source ↗ |
| MiniMax M3 | 24.8% | 2026-07-20 | independent | Source ↗ |
| Qwen3-235B | 25.5% | 2026-07-13 | independent | Source ↗ |
| QwQ Plus | 25.9% | 2026-07-13 | independent | Source ↗ |
| Z.ai: GLM 5.2 | 22.1% | 2026-07-11 | independent | Source ↗ |
| Grok 4 | 26.6% | 2026-07-10 | independent | Source ↗ |
| GLM-5.2 | 25.3% | 2026-07-07 | independent | Source ↗ |
| MoonshotAI: Kimi K2.7 Code | 23% | 2026-07-07 | independent | Source ↗ |
| Claude Fable 5 | 26.9% | 2026-07-04 | independent | Source ↗ |
| Claude Fable 5 | 23.8% | 2026-07-04 | independent | Source ↗ |
| Nemotron 3 Ultra | 23.5% | 2026-06-29 | independent | Source ↗ |
| Nemotron 3.5 Content Safety | 20% | 2026-06-29 | independent | Source ↗ |
| Nemotron 3 Ultra | 22.1% | 2026-06-29 | independent | Source ↗ |
| Qwen: Qwen3.7 Plus | 22.1% | 2026-06-27 | independent | Source ↗ |
| MiniMax: MiniMax M3 | 22.1% | 2026-06-25 | independent | Source ↗ |
| Gemini 3.1 Flash Image | 22.1% | 2026-06-22 | independent | Source ↗ |
| Gemini 3 Pro Image (Nano Banana Pro) | 23.8% | 2026-06-22 | independent | Source ↗ |
| Claude Opus 4.8 | 23.8% | 2026-06-22 | independent | Source ↗ |
| Hy-MT2-1.8B | 20% | 2026-06-15 | independent | Source ↗ |
| Hy-MT2-30B-A3B | 21.1% | 2026-06-15 | independent | Source ↗ |
| Hy-MT2-7B | 21.1% | 2026-06-15 | independent | Source ↗ |
| Qwen: Qwen3.7 Max | 23% | 2026-06-14 | independent | Source ↗ |
| SpaceXAI: Grok Build 0.1 | 22.1% | 2026-06-14 | independent | Source ↗ |
| Gemini 3.5 Flash | 23% | 2026-06-13 | independent | Source ↗ |
| Gemini 3 Pro | 25.8% | 2026-06-12 | independent | Source ↗ |
| Gemini 3.1 Flash Lite | 22.1% | 2026-06-01 | independent | Source ↗ |
| GPT Chat Latest | 23.8% | 2026-05-30 | independent | Source ↗ |
| Command A+ | 22.1% | 2026-05-26 | independent | Source ↗ |
| Mistral Medium 3.5 | 23% | 2026-05-25 | independent | Source ↗ |
| SpaceXAI: Grok 4.3 | 22.1% | 2026-05-25 | independent | Source ↗ |
| Qwen: Qwen3.6 Flash | 22.1% | 2026-05-22 | independent | Source ↗ |
| GPT-5.5 Pro | 23.8% | 2026-05-18 | independent | Source ↗ |
| GPT-5.5 | 23.8% | 2026-05-18 | independent | Source ↗ |
| DeepSeek V4 Pro 0423 | 22.1% | 2026-05-18 | independent | Source ↗ |
| DeepSeek V4 Flash 0423 | 20% | 2026-05-18 | independent | Source ↗ |
| MiMo-V2.5 | 22.8% | 2026-05-17 | independent | Source ↗ |
| Xiaomi: MiMo-V2.5-Pro | 21.1% | 2026-05-17 | independent | Source ↗ |
| Xiaomi: MiMo-V2.5 | 21.1% | 2026-05-17 | independent | Source ↗ |
| Qwen: Qwen3.6 27B | 22.1% | 2026-05-17 | independent | Source ↗ |
| Hy3 | 21.1% | 2026-05-16 | independent | Source ↗ |
| GPT-5.4 Image 2 | 23.8% | 2026-05-16 | independent | Source ↗ |
| Qwen: Qwen3.5 Plus 2026-04-20 | 22.1% | 2026-05-15 | independent | Source ↗ |
| MoonshotAI: Kimi K2.6 | 23% | 2026-05-15 | independent | Source ↗ |
| Claude Opus 4.7 | 23.8% | 2026-05-11 | independent | Source ↗ |
| Qwen: Qwen3.6 35B A3B | 21.1% | 2026-05-10 | independent | Source ↗ |
| Z.ai: GLM 5.1 | 23% | 2026-05-01 | independent | Source ↗ |
| Gemma 4 26B A4B | 21.1% | 2026-04-28 | independent | Source ↗ |
| Gemma 4 31B | 21.1% | 2026-04-27 | independent | Source ↗ |
| Qwen: Qwen3.6 Plus | 22.1% | 2026-04-27 | independent | Source ↗ |
| Z.ai: GLM 5V Turbo | 23% | 2026-04-26 | independent | Source ↗ |
| Lyria 3 Pro | 20% | 2026-04-24 | independent | Source ↗ |
| Lyria 3 Clip | 20% | 2026-04-24 | independent | Source ↗ |
| Reka Edge | 20% | 2026-04-14 | independent | Source ↗ |
| MiniMax: MiniMax M2.7 | 22.1% | 2026-04-12 | independent | Source ↗ |
| GPT-5.4 Nano | 22.1% | 2026-04-11 | independent | Source ↗ |
| GPT-5.4 Mini | 23% | 2026-04-11 | independent | Source ↗ |
| Z.ai: GLM 5 Turbo | 23% | 2026-04-09 | independent | Source ↗ |
| Qwen: Qwen3.5-9B | 21.1% | 2026-04-04 | independent | Source ↗ |
| SpaceXAI: Grok 4.20 Multi-Agent | 22.1% | 2026-04-03 | independent | Source ↗ |
| SpaceXAI: Grok 4.20 | 22.1% | 2026-04-03 | independent | Source ↗ |
| ByteDance Seed: Seed-2.0-Lite | 22.1% | 2026-04-03 | independent | Source ↗ |
| GPT-5.4 Pro | 23.8% | 2026-03-30 | independent | Source ↗ |
| GPT-5.4 | 23.8% | 2026-03-30 | independent | Source ↗ |
| Gemini 3.1 Flash Lite | 22.1% | 2026-03-28 | independent | Source ↗ |
| Gemini 3.1 Flash Image | 22.1% | 2026-03-23 | independent | Source ↗ |
| GPT-5.3-Codex | 23.8% | 2026-03-21 | independent | Source ↗ |
| ByteDance Seed: Seed-2.0-Mini | 21.1% | 2026-03-21 | independent | Source ↗ |
| Qwen: Qwen3.5-35B-A3B | 21.1% | 2026-03-21 | independent | Source ↗ |
| Qwen: Qwen3.5-27B | 22.1% | 2026-03-21 | independent | Source ↗ |
| Qwen: Qwen3.5-122B-A10B | 22.1% | 2026-03-21 | independent | Source ↗ |
| Qwen: Qwen3.5-Flash | 21.1% | 2026-03-21 | independent | Source ↗ |
| Gemini 3.1 Pro Custom Tools | 23.8% | 2026-03-16 | independent | Source ↗ |
| Gemini 3.1 Pro | 23.8% | 2026-03-16 | independent | Source ↗ |
| Claude Sonnet 4.6 | 23.8% | 2026-03-14 | independent | Source ↗ |
| Qwen: Qwen3.5 Plus 2026-02-15 | 22.1% | 2026-03-13 | independent | Source ↗ |
| Qwen: Qwen3.5 397B A17B | 23% | 2026-03-13 | independent | Source ↗ |
| Z.ai: GLM 5 | 23% | 2026-03-08 | independent | Source ↗ |
| MiniMax: MiniMax M2.5 | 21.1% | 2026-03-08 | independent | Source ↗ |
| Claude Opus 4.6 | 23.8% | 2026-03-02 | independent | Source ↗ |
| MoonshotAI: Kimi K2.5 | 22.1% | 2026-02-21 | independent | Source ↗ |
| MiniMax: MiniMax M2-her | 22.1% | 2026-02-17 | independent | Source ↗ |
| Z.ai: GLM 4.7 Flash | 21.1% | 2026-02-13 | independent | Source ↗ |
| GPT-5.2-Codex | 23.8% | 2026-02-08 | independent | Source ↗ |
| MiniMax: MiniMax M2.1 | 22.1% | 2026-01-17 | independent | Source ↗ |
| Z.ai: GLM 4.7 | 22.1% | 2026-01-16 | independent | Source ↗ |
| o3 | 26.2% | 2026-01-14 | independent | Source ↗ |
| Gemini 3 Flash | 22.1% | 2026-01-11 | independent | Source ↗ |
| GPT-5.2 Chat | 23.8% | 2026-01-05 | independent | Source ↗ |
| GPT-5.2 Pro | 23.8% | 2026-01-05 | independent | Source ↗ |
| GPT-5.2 | 23.8% | 2026-01-05 | independent | Source ↗ |
| Z.ai: GLM 4.6V | 21.1% | 2026-01-02 | independent | Source ↗ |
| GPT-5.1-Codex-Max | 23% | 2025-12-29 | independent | Source ↗ |
| DeepSeek V3.2 | 21.1% | 2025-12-26 | independent | Source ↗ |
| Claude Opus 4.5 | 23.8% | 2025-12-19 | independent | Source ↗ |
| Gemini 3 Pro Image (Nano Banana Pro) | 23.8% | 2025-12-15 | independent | Source ↗ |
| GPT-5.1 | 23% | 2025-12-08 | independent | Source ↗ |
| GPT-5.1-Codex | 23% | 2025-12-08 | independent | Source ↗ |
| GPT-5.1-Codex-Mini | 22.1% | 2025-12-08 | independent | Source ↗ |
| Mistral Large 3 | 23.5% | 2025-12-07 | independent | Source ↗ |
| MoonshotAI: Kimi K2 Thinking | 22.1% | 2025-12-01 | independent | Source ↗ |
| MiniMax: MiniMax M2 | 22.1% | 2025-11-17 | independent | Source ↗ |
| Qwen: Qwen3 VL 8B Instruct | 21.1% | 2025-11-08 | independent | Source ↗ |
| GPT-5 Pro | 23.8% | 2025-10-31 | independent | Source ↗ |
| Qwen: Qwen3 VL 30B A3B Thinking | 21.1% | 2025-10-31 | independent | Source ↗ |
| Qwen: Qwen3 VL 30B A3B Instruct | 21.1% | 2025-10-31 | independent | Source ↗ |
| Claude Haiku 4.5 | 23% | 2025-10-26 | independent | Source ↗ |
| Claude Sonnet 4.5 | 23.8% | 2025-10-24 | independent | Source ↗ |
| Qwen: Qwen3 VL 235B A22B Instruct | 22.1% | 2025-10-18 | independent | Source ↗ |
| Qwen: Qwen3 Coder Flash | 21.1% | 2025-10-12 | independent | Source ↗ |
| Qwen: Qwen3 Next 80B A3B Instruct | 22.1% | 2025-10-06 | independent | Source ↗ |
| MoonshotAI: Kimi K2 0905 | 22.1% | 2025-09-29 | independent | Source ↗ |
| GPT-5 | 23% | 2025-09-01 | independent | Source ↗ |
| GPT-5 Mini | 22.1% | 2025-09-01 | independent | Source ↗ |
| GPT-5 Nano | 21.1% | 2025-09-01 | independent | Source ↗ |
| Claude 4.1 Opus | 23.8% | 2025-08-30 | independent | Source ↗ |
| Qwen: Qwen3 Coder 30B A3B Instruct | 21.1% | 2025-08-25 | independent | Source ↗ |
| Qwen: Qwen3 30B A3B Instruct 2507 | 21.1% | 2025-08-23 | independent | Source ↗ |
| Qwen: Qwen3 235B A22B Thinking 2507 | 23% | 2025-08-19 | independent | Source ↗ |
| Qwen: Qwen3 Coder 480B A35B | 21.1% | 2025-08-17 | independent | Source ↗ |
| ByteDance: UI-TARS 7B | 20% | 2025-08-16 | independent | Source ↗ |
| Qwen: Qwen3 235B A22B Instruct 2507 | 21.1% | 2025-08-15 | independent | Source ↗ |
| MoonshotAI: Kimi K2 0711 | 22.1% | 2025-08-05 | independent | Source ↗ |
| ByteDance Seed: Seed 1.6 Flash | 21.1% | 2025-07-20 | independent | Source ↗ |
| ByteDance Seed: Seed 1.6 | 22.1% | 2025-07-20 | independent | Source ↗ |
| MiniMax: MiniMax M1 | 22.1% | 2025-07-12 | independent | Source ↗ |
| Sarvam 105B | 23.7% | 2025-07-10 | independent | Source ↗ |
| o3 Pro | 23.8% | 2025-07-05 | independent | Source ↗ |
| Sarvam Vision (Document AI) | 21% | 2025-06-14 | independent | Source ↗ |
| Amazon Nova 2 Lite | 22.3% | 2025-06-12 | independent | Source ↗ |
| Amazon Nova 2 Lite | 21.3% | 2025-06-12 | independent | Source ↗ |
| Qwen: Qwen3 30B A3B | 21.1% | 2025-05-23 | independent | Source ↗ |
| Qwen: Qwen3 14B | 21.1% | 2025-05-23 | independent | Source ↗ |
| Qwen: Qwen3 32B | 21.1% | 2025-05-23 | independent | Source ↗ |
| o4 Mini High | 23% | 2025-05-11 | independent | Source ↗ |
| o3 | 23% | 2025-05-11 | independent | Source ↗ |
| o4 Mini | 23% | 2025-05-11 | independent | Source ↗ |
| Muse Wonder | 25.5% | 2025-05-10 | independent | Source ↗ |
| Muse Wonder Full | 26.5% | 2025-05-10 | independent | Source ↗ |
| GPT-4.1 | 23% | 2025-05-09 | independent | Source ↗ |
| GPT-4.1 Mini | 22.1% | 2025-05-09 | independent | Source ↗ |
| GPT-4.1 Nano | 21.1% | 2025-05-09 | independent | Source ↗ |
| Gemma 3 27B | 23.5% | 2025-05-07 | independent | Source ↗ |
| Nemotron 3 Ultra 550B | 25.3% | 2025-05-05 | independent | Source ↗ |
| Sarvam 30B | 22.4% | 2025-05-05 | independent | Source ↗ |
| Llama 4 Maverick | 25.1% | 2025-04-30 | independent | Source ↗ |
| Llama 4 Scout | 24.4% | 2025-04-30 | independent | Source ↗ |
| Gemini 2.5 Pro | 25.1% | 2025-04-19 | independent | Source ↗ |
| Gemini 2.5 Pro | 23% | 2025-04-19 | independent | Source ↗ |
| Gemini 2.5 Pro 06-05 | 23% | 2025-04-19 | independent | Source ↗ |
| Microsoft-Decision-1 | 23.9% | 2025-04-16 | independent | Source ↗ |
| Nemotron 3 Super 120B | 23.5% | 2025-04-14 | independent | Source ↗ |
| OpenAI Decisions API | 25.2% | 2025-04-12 | independent | Source ↗ |
| Jev 1.14 | 24.1% | 2025-04-08 | independent | Source ↗ |
| Krutrim-2 | 23.1% | 2025-04-06 | independent | Source ↗ |
| Reka Flash 3 | 20% | 2025-04-06 | independent | Source ↗ |
| Amazon Nova Premier 1.0 | 24.9% | 2025-04-04 | independent | Source ↗ |
| Amazon: Nova Premier 1.0 | 23.8% | 2025-04-04 | independent | Source ↗ |
| QwQ 32B | 24.9% | 2025-03-30 | independent | Source ↗ |
| Saaras v3 (Speech-to-Text) | 18.2% | 2025-03-26 | independent | Source ↗ |
| Bulbul v3 (Text-to-Speech) | 16.8% | 2025-03-26 | independent | Source ↗ |
| Phi-4-multimodal | 22.7% | 2025-03-25 | independent | Source ↗ |
| GPT-4.5 | 22.4% | 2025-03-24 | independent | Source ↗ |
| Claude 3.7 Sonnet | 25.5% | 2025-03-21 | independent | Source ↗ |
| Grok 3 | 25.8% | 2025-03-14 | independent | Source ↗ |
| Grok 3 Mini | 23.8% | 2025-03-14 | independent | Source ↗ |
| Phi-4-mini | 22% | 2025-03-12 | independent | Source ↗ |
| Perplexity: Sonar Deep Research | 23% | 2025-03-11 | independent | Source ↗ |
| Gemini 2.5 Flash (Thinking) | 24.4% | 2025-03-07 | independent | Source ↗ |
| Perplexity: Sonar Reasoning Pro | 23% | 2025-03-03 | independent | Source ↗ |
| Gemini 2.0 Pro | 23.8% | 2025-03-02 | independent | Source ↗ |
| Gemini 2.0 Flash | 22.1% | 2025-03-02 | independent | Source ↗ |
| Sonar Reasoning Pro | 24.9% | 2025-03-02 | independent | Source ↗ |
| OpenAI o3-mini (High) | 11% | 2025-02-28 | independent | Source ↗ |
| Qwen: Qwen3 Coder Next | 21.1% | 2025-02-28 | independent | Source ↗ |
| o3-mini | 25.3% | 2025-02-25 | independent | Source ↗ |
| o3 Mini High | 23% | 2025-02-25 | independent | Source ↗ |
| o3 Mini | 23% | 2025-02-25 | independent | Source ↗ |
| Qwen 2.5 Max | 23.2% | 2025-02-22 | independent | Source ↗ |
| Qwen: Qwen2.5 VL 72B Instruct | 21.1% | 2025-02-20 | independent | Source ↗ |
| Qwen: Qwen-Plus | 21.1% | 2025-02-19 | independent | Source ↗ |
| Ollama DeepSeek-R1 (Q4_K_M) | 25.5% | 2025-02-16 | independent | Source ↗ |
| Perplexity: Sonar Pro Search | 23.8% | 2025-02-15 | independent | Source ↗ |
| Perplexity: Sonar Pro | 23.8% | 2025-02-15 | independent | Source ↗ |
| DeepSeek-R1 | 25.8% | 2025-02-14 | independent | Source ↗ |
| Gemini 2.0 Flash Thinking | 24.8% | 2025-02-14 | independent | Source ↗ |
| Kimi k1.5 | 24.1% | 2025-02-14 | independent | Source ↗ |
| R1 0528 | 22.1% | 2025-02-14 | independent | Source ↗ |
| MiniMax-Text-01 | 22.1% | 2025-02-09 | independent | Source ↗ |
| MiniMax: MiniMax-01 | 22.1% | 2025-02-09 | independent | Source ↗ |
| Codestral 25.01 | 19.9% | 2025-02-08 | independent | Source ↗ |
| Phi-4 | 24.2% | 2025-02-02 | independent | Source ↗ |
| Phi 4 | 20% | 2025-02-02 | independent | Source ↗ |
| DeepSeek-V3 | 22.1% | 2025-01-20 | independent | Source ↗ |
| DeepSeek V3.2 Exp | 21.1% | 2025-01-20 | independent | Source ↗ |
| DeepSeek V3.1 Terminus | 21.1% | 2025-01-20 | independent | Source ↗ |
| DeepSeek V3.1 | 21.1% | 2025-01-20 | independent | Source ↗ |
| DeepSeek V3 0324 | 22.1% | 2025-01-20 | independent | Source ↗ |
| DeepSeek V3 | 22.1% | 2025-01-20 | independent | Source ↗ |
| o1 | 23.8% | 2025-01-11 | independent | Source ↗ |
| Sonar Pro | 23% | 2025-01-04 | independent | Source ↗ |
| Ollama Llama 3.3 (Q4_K_M) | 23.1% | 2025-01-02 | independent | Source ↗ |
| Llama 3.3 70B | 21.3% | 2024-12-31 | independent | Source ↗ |
| Llama 3.3 70B Instruct | 23.4% | 2024-12-31 | independent | Source ↗ |
| Llama 3.3 70B Instruct | 21.1% | 2024-12-31 | independent | Source ↗ |
| o1 | 25% | 2024-12-30 | independent | Source ↗ |
| o1-pro | 23.8% | 2024-12-30 | independent | Source ↗ |
| Amazon Nova Pro | 22.5% | 2024-12-28 | independent | Source ↗ |
| Amazon Nova Lite | 20.2% | 2024-12-28 | independent | Source ↗ |
| Amazon Nova Pro 1.0 | 23.2% | 2024-12-28 | independent | Source ↗ |
| Amazon Nova Lite 1.0 | 21.3% | 2024-12-28 | independent | Source ↗ |
| Amazon Nova Micro 1.0 | 19.3% | 2024-12-28 | independent | Source ↗ |
| Amazon: Nova Lite 1.0 | 21.1% | 2024-12-28 | independent | Source ↗ |
| Amazon: Nova Micro 1.0 | 20% | 2024-12-28 | independent | Source ↗ |
| Amazon: Nova Pro 1.0 | 23% | 2024-12-28 | independent | Source ↗ |
| Command R7B (12-2024) | 20% | 2024-12-26 | independent | Source ↗ |
| QwQ-32B | 23.9% | 2024-12-23 | independent | Source ↗ |
| Krutrim Pro | 22% | 2024-12-15 | independent | Source ↗ |
| GPT-4o | 23% | 2024-12-15 | independent | Source ↗ |
| Upstage: Solar Pro 4 | 21.1% | 2024-12-15 | independent | Source ↗ |
| Jev | 23.8% | 2024-12-10 | independent | Source ↗ |
| Sonar | 22% | 2024-12-10 | independent | Source ↗ |
| TypeSafe: Jev Router | 20% | 2024-12-10 | independent | Source ↗ |
| Perplexity: Sonar | 21.1% | 2024-12-10 | independent | Source ↗ |
| Qwen 2.5 Coder 32B | 21.3% | 2024-12-07 | independent | Source ↗ |
| Qwen2.5 Coder 32B Instruct | 21.1% | 2024-12-07 | independent | Source ↗ |
| Hunyuan-Large | 23.1% | 2024-11-30 | independent | Source ↗ |
| Claude 3.5 Haiku | 19.6% | 2024-11-29 | independent | Source ↗ |
| Claude 3.5 Sonnet | 22% | 2024-11-16 | independent | Source ↗ |
| Claude 3.5 Sonnet (v2) | 23% | 2024-11-16 | independent | Source ↗ |
| BharatGPT-14B | 21% | 2024-11-12 | independent | Source ↗ |
| Yi-Lightning | 22.7% | 2024-11-09 | independent | Source ↗ |
| Llama-3.1-Nemotron-70B-Instruct | 23.8% | 2024-11-09 | independent | Source ↗ |
| best frontier model | 2% | 2024-11-07 | independent | Source ↗ |
| Llama 3.2 1B Instruct | 21.1% | 2024-10-20 | independent | Source ↗ |
| Llama 3.2 3B Instruct | 21.1% | 2024-10-20 | independent | Source ↗ |
| Qwen 2.5 72B Instruct | 24.1% | 2024-10-14 | independent | Source ↗ |
| Qwen: Qwen2.5 7B Instruct | 20% | 2024-10-14 | independent | Source ↗ |
| Qwen2.5 72B Instruct | 21.1% | 2024-10-14 | independent | Source ↗ |
| Switchyard | 20% | 2024-10-10 | independent | Source ↗ |
| Mistral Small 4 | 21.1% | 2024-10-10 | independent | Source ↗ |
| GPT Audio | 23% | 2024-10-10 | independent | Source ↗ |
| GPT Audio Mini | 22.1% | 2024-10-10 | independent | Source ↗ |
| Nemotron 3 Nano 30B A3B | 21.1% | 2024-10-10 | independent | Source ↗ |
| Devstral 2 2512 | 22.1% | 2024-10-10 | independent | Source ↗ |
| Ministral 3 14B 2512 | 20% | 2024-10-10 | independent | Source ↗ |
| Ministral 3 8B 2512 | 20% | 2024-10-10 | independent | Source ↗ |
| Ministral 3 3B 2512 | 20% | 2024-10-10 | independent | Source ↗ |
| Voxtral Small 24B 2507 | 21.1% | 2024-10-10 | independent | Source ↗ |
| gpt-oss-safeguard-20b | 21.1% | 2024-10-10 | independent | Source ↗ |
| Gemini 2.5 Flash Image | 22.1% | 2024-10-10 | independent | Source ↗ |
| Z.ai: GLM 4.6 | 22.1% | 2024-10-10 | independent | Source ↗ |
| Mistral Medium 3.1 | 22.1% | 2024-10-10 | independent | Source ↗ |
| Z.ai: GLM 4.5V | 22.1% | 2024-10-10 | independent | Source ↗ |
| gpt-oss-120b | 21.1% | 2024-10-10 | independent | Source ↗ |
| gpt-oss-20b | 20% | 2024-10-10 | independent | Source ↗ |
| Codestral 2508 | 21.1% | 2024-10-10 | independent | Source ↗ |
| Z.ai: GLM 4.5 | 22.1% | 2024-10-10 | independent | Source ↗ |
| Z.ai: GLM 4.5 Air | 21.1% | 2024-10-10 | independent | Source ↗ |
| Gemini 2.5 Flash Lite | 21.1% | 2024-10-10 | independent | Source ↗ |
| Hunyuan A13B Instruct | 21.1% | 2024-10-10 | independent | Source ↗ |
| Mistral Small 3.2 24B | 21.1% | 2024-10-10 | independent | Source ↗ |
| Gemini 2.5 Flash | 22.1% | 2024-10-10 | independent | Source ↗ |
| Mistral Medium 3 | 22.1% | 2024-10-10 | independent | Source ↗ |
| Llama Guard 4 12B | 20% | 2024-10-10 | independent | Source ↗ |
| Llama 4 Maverick | 21.1% | 2024-10-10 | independent | Source ↗ |
| Llama 4 Scout | 21.1% | 2024-10-10 | independent | Source ↗ |
| Mistral Small 3.1 24B | 21.1% | 2024-10-10 | independent | Source ↗ |
| Gemma 3 4B | 20% | 2024-10-10 | independent | Source ↗ |
| Gemma 3 12B | 20% | 2024-10-10 | independent | Source ↗ |
| Gemma 3 27B | 21.1% | 2024-10-10 | independent | Source ↗ |
| Saba | 21.1% | 2024-10-10 | independent | Source ↗ |
| Mistral Small 3 | 20% | 2024-10-10 | independent | Source ↗ |
| Mistral Nemo | 20% | 2024-10-10 | independent | Source ↗ |
| Gemma 2 27B | 21.1% | 2024-10-10 | independent | Source ↗ |
| Mixtral 8x22B Instruct | 23% | 2024-10-10 | independent | Source ↗ |
| GPT-4 Turbo | 23.8% | 2024-10-10 | independent | Source ↗ |
| Mistral Large | 23% | 2024-10-10 | independent | Source ↗ |
| GPT-3.5 Turbo (older v0613) | 22.1% | 2024-10-10 | independent | Source ↗ |
| GPT-3.5 Turbo Instruct | 22.1% | 2024-10-10 | independent | Source ↗ |
| GPT-3.5 Turbo 16k | 23% | 2024-10-10 | independent | Source ↗ |
| GPT-3.5 Turbo | 22.1% | 2024-10-10 | independent | Source ↗ |
| GPT-4 | 23.8% | 2024-10-10 | independent | Source ↗ |
| GLM-4-Plus | 21.8% | 2024-09-14 | independent | Source ↗ |
| Sarvam 2B (Sarvam-1) | 20.2% | 2024-09-08 | independent | Source ↗ |
| GPT-4o | 23% | 2024-08-31 | independent | Source ↗ |
| Command R (08-2024) | 21.1% | 2024-08-26 | independent | Source ↗ |
| Gemma 2 2B | 15.4% | 2024-08-25 | independent | Source ↗ |
| Mistral Large 2 | 20.2% | 2024-08-18 | independent | Source ↗ |
| Mistral Large 2 | 22% | 2024-08-18 | independent | Source ↗ |
| Mistral Large 3 2512 | 22.1% | 2024-08-18 | independent | Source ↗ |
| Llama 3.1 405B | 21.7% | 2024-08-17 | independent | Source ↗ |
| Llama 3.1 70B Instruct | 21.1% | 2024-08-17 | independent | Source ↗ |
| Llama 3.1 8B Instruct | 20% | 2024-08-17 | independent | Source ↗ |
| GPT-4o mini | 19% | 2024-08-12 | independent | Source ↗ |
| GPT-4o-mini | 21.1% | 2024-08-12 | independent | Source ↗ |
| GPT-4o-mini | 21.1% | 2024-08-12 | independent | Source ↗ |
| ERNIE 4.0 Turbo | 21.6% | 2024-07-23 | independent | Source ↗ |
| Gemma 2 27B | 21.3% | 2024-07-22 | independent | Source ↗ |
| Gemma 2 9B | 19.5% | 2024-07-22 | independent | Source ↗ |
| Nemotron-4-340B-Instruct | 23% | 2024-07-09 | independent | Source ↗ |
| Nemotron-4-340B-Reward | 23.5% | 2024-07-09 | independent | Source ↗ |
| Hanooman 7B | 20.4% | 2024-06-09 | independent | Source ↗ |
| GPT-4o | 21.5% | 2024-06-07 | independent | Source ↗ |
| GPT-4o | 23% | 2024-06-07 | independent | Source ↗ |
| GPT-4o | 23.8% | 2024-06-07 | independent | Source ↗ |
| Snowflake Arctic | 20.2% | 2024-05-19 | independent | Source ↗ |
| WizardLM-2 8x22B | 22.4% | 2024-05-10 | independent | Source ↗ |
| DBRX Instruct | 20.4% | 2024-04-21 | independent | Source ↗ |
| Airavata 7B | 19.3% | 2024-02-19 | independent | Source ↗ |
| IndicTrans-2 | 16.8% | 2023-10-10 | independent | Source ↗ |
| Nemotron 3 Super | 21.1% | 2023-04-05 | independent | Source ↗ |
Human Baseline & Difficulty Horizon
Calibrated human reference points, specialist benchmarks, and ceiling thresholds.Elite mathematics competition winners and PhD researchers score <2% without computing tools, ~25% with Python.
Metric & Scoring Methodology
Verification protocols, aggregation formulas, and specialized metric variants.accuracy — % of problems solved, automated answer verification (%)Dataset & Compute Cost
Evaluation volume, public availability, API pricing, and local hardware requirements.$5 – $20 USD for full benchmark evaluation run on frontier APIs.
1x NVIDIA RTX 4090 (24GB) or A100 (40GB/80GB) via vLLM / SGLang
How to Run & Reproduce
Standardized evaluation protocols, CLI commands, and reproducible runner templates.lm_eval --model hf --model_args pretrained=<model_path> --tasks frontiermath --batch_size autoopencompass --datasets frontiermath --models <model_config># Standard API Evaluation Loop
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
temperature=0.0,
)When publishing results for FrontierMath, always report the exact prompt template, few-shot exemplar ordering, sampling temperature (temperature=0), maximum reasoning budget tokens, and the precise timestamped model snapshot ID.
Contamination & Memorization Analysis
Audit of pretraining exposure risks, memorization vectors, and refresh policies.Static fixed snapshot
Public on web / HuggingFace