Related Model — Select Model (318 Available) — Claude Fable 5 GPT-5.6 Sol Claude Fable 5.1 (Hybrid Reasoning) Claude Opus 5 GPT-6 Astra (max) GPT-5.5 Thinking GPT-5 DeepSeek V4.1 Flash Kimi K3 Grok 4 Gemini 3.7 Flash Grok 4.6 Claude Sonnet 5 Gemini 3 Pro Claude 3.7 Sonnet DeepSeek V4-Pro o3 Gemini 3.6 Flash Qwen3-235B Gemini 2.5 Pro DeepSeek-R1 QwQ Plus GLM-5.2 Llama 4 Maverick Sonar Reasoning Pro Ollama DeepSeek-R1 (Q4_K_M) Grok 3 MiniMax M3 o3-mini Llama 4 Scout DeepSeek V4-Flash Gemini 2.0 Flash Thinking Gemini 3.5 Flash-Lite o1 GPT-4.5 Gemini 2.0 Pro QwQ 32B Mistral Large 3 Qwen 2.5 Max Qwen 2.5 72B Instruct Claude 3.5 Sonnet Llama 3.3 70B Instruct Hunyuan-Large Gemma 3 27B Preview Mistral Large 2 Kimi k1.5 Sonar Pro Ollama Llama 3.3 (Q4_K_M) Yi-Lightning Amazon Nova Pro DeepSeek-V3 Gemini 2.0 Flash MiniMax-Text-01 Llama 3.1 405B Qwen 2.5 Coder 32B ERNIE 4.0 Turbo GPT-4o Sonar Large GLM-4-Plus Llama 3.3 70B Reka Core Mistral Large 2 Codestral 25.01 Gemma 2 27B Solar Pro Claude 3 Opus Gemini 1.5 Pro DBRX Instruct Claude 3.5 Haiku Snowflake Arctic Phi-4 Amazon Nova Lite Command R+ GPT-4o mini Gemini 1.5 Flash QwQ-32B Preview Gemini 2.5 Flash (Thinking) Claude 3.5 Sonnet (v2) Sonar Gemma 2 9B Gemma 2 2B GPT-4.5 (pure LLM) r1 / r1-zero (single CoT) ARChitects (Kaggle 2024 winner) o3-mini high (Epoch independent eval, Tiers 1–3, tools-on) Goedel-Prover (Lean, 7/644, pass@512) o3-mini (CodeSOTA MBPP pass@1) DeepSeek-V3 (base, 0-shot) DeepSeek V3 (open weight) DeepSeek-V3 (CodeSOTA MBPP pass@1) o3-preview-low (CoT + search/synthesis) o3 (AIME 2024, pass@1) Jeremy Berman system Qwen2.5-Coder-32B-Instruct (EvalPlus, greedy, HumanEval base pass@1) Qwen2.5-Coder-32B-Instruct Qwen2.5-Coder-32B-Instruct (CodeSOTA MBPP pass@1) Qwen2.5-Coder 32B (CodeSOTA MBPP pass@1) best frontier model (Tiers 1–3, tools-on) Grok Beta DeepSeek-V3 (November 2024) DeepSeek-V2.5 (November 2024) Claude 3.5 Sonnet (Oct 2024; CodeSOTA MBPP pass@1) Claude 3.5 Sonnet (upgraded; Anthropic tools scaffold) OpenAI o1-mini Gemini 1.5 Pro 002 Qwen2.5 72B (base, 0-shot) o1-preview GPT-4o (AIME 2024, pass@1) o1 (AIME 2024, pass@1) o1-preview (September 2024) o1-mini (September 2024) o1 (MATH-500) O1 Preview (Sept 2024) O1 Mini (Sept 2024) GPT-4o (Agentless) GPT-4o (August 2024) Jamba-1.5-large (README Avg. snapshot) Gemini-1.5-pro (README Avg. snapshot) Qwen2.5-14B-Instruct-1M (README Avg. snapshot) Qwen3-235B-A22B (README Avg. snapshot) Qwen3-14B (README Avg. snapshot) Jamba-1.5-mini (README Avg. snapshot) Qwen3-32B (README Avg. snapshot) EXAONE-4.0-32B (README Avg. snapshot) Qwen2.5-7B-Instruct-1M (README Avg. snapshot) GPT-4-1106-preview (README Avg. snapshot) Claude Fable 5 (LLM Stats snapshot) Claude Mythos Preview (LLM Stats snapshot) Claude Opus 4.8 (LLM Stats snapshot) Grok 4.5 (LLM Stats snapshot) GPT-5.6 Sol (LLM Stats snapshot) Claude Opus 4.7 (LLM Stats snapshot) GPT-5.6 Terra (LLM Stats snapshot) Claude Sonnet 5 (LLM Stats snapshot) GPT-5.6 Luna (LLM Stats snapshot) GLM-5.2 (LLM Stats snapshot) Muse Spark 1.1 (LLM Stats snapshot) Qwen3.7 Max (LLM Stats snapshot) MiniMax M3 (LLM Stats snapshot) Gemini 3.6 Flash (LLM Stats snapshot) Kimi K2.6 (LLM Stats snapshot) GPT-5.5 (LLM Stats snapshot) GLM-5.1 (LLM Stats snapshot) Llama 3.1 405B (5-shot) GPT-4o Mini (July 2024) GPT-4o (paper era, Lean, ~1/640) Claude 3.5 Sonnet (3-shot CoT) Claude 3.5 Sonnet (June 2024) DeepSeek-Coder-V2-Instruct DeepSeek-Coder-V2-Instruct (CodeSOTA MBPP pass@1) o1-mini Claude 4.7 (High) GPT-5.5 Pro (High) GPT-5.6 Sol (xHigh) NVARC Claude 4.7 (Max) GPT-5.5 (xHigh) GPT-5.6 Sol (Max) Anthropic Opus 4.6 (Max) Gemini 3.1 Pro (Preview) GPT-5.5 (High) Claude Opus 4.8 (High) SWE-1.6 (FrontierCode 1.1 Main pass rate) Kimi K2.7 Code (FrontierCode 1.1 Main pass rate) SWE-1.7 (FrontierCode 1.1 Main pass rate) GPT-5.5 (FrontierCode 1.1 Main pass rate) Claude Opus 4.8 (FrontierCode 1.1 Main pass rate) Qwen3.7 Max (qwen3-7-max; closed) Qwen3.7 Plus (qwen3-7-plus; closed) GLM-4.7 (glm-4-7; open weight) Qwen3.6-27B (open weight) Qwen3.6-35B-A3B (open weight) Gemini-2.5-Pro (paper Overall) GPT-5 (paper Overall) Qwen3-235B-A22B-Thinking-2507 (paper Overall) Qwen3-Next-80B-A3B-Thinking (paper Overall) DeepSeek-R1-0528 (paper Overall) DeepSeek-R1 (paper Overall) Qwen3-30B-A3B-Thinking-2507 (paper Overall) Claude-4-Sonnet (paper Overall) Gemini-2.5-Flash (paper Overall) MiniMax-M2 (paper Overall) GPT-5.5 (xhigh, expected performance) GPT-5.6-Sol (max, expected performance) o4-mini (CodeSOTA MBPP pass@1) Claude Opus 4 (CodeSOTA MBPP pass@1) GPT-4.1 (CodeSOTA MBPP pass@1) Claude Sonnet 4 (CodeSOTA MBPP pass@1) GPT-5.6 Sol (LLM Stats MRCR v2 8-needle) GPT-5.6 Terra (LLM Stats MRCR v2 8-needle) Claude Opus 4.6 (LLM Stats MRCR v2 8-needle) GPT-5.5 (LLM Stats MRCR v2 8-needle) Gemma 4 31B (LLM Stats MRCR v2 8-needle) Gemini 3.1 Flash-Lite (LLM Stats MRCR v2 8-needle) Gemini 3.6 Flash (LLM Stats MRCR v2 8-needle) Gemma 4 26B-A4B (LLM Stats MRCR v2 8-needle) Gemma 4 12B (LLM Stats MRCR v2 8-needle) GPT-5.6 Luna (LLM Stats MRCR v2 8-needle) GPT-5.4 mini (LLM Stats MRCR v2 8-needle) GPT-5.4 nano (LLM Stats MRCR v2 8-needle) Gemini 3.5 Flash (LLM Stats MRCR v2 8-needle) Gemini 3.1 Pro (LLM Stats MRCR v2 8-needle) Gemini 3 Pro (LLM Stats MRCR v2 8-needle) Gemma 4 E4B (LLM Stats MRCR v2 8-needle) Gemini 3 Flash (LLM Stats MRCR v2 8-needle) Gemini 3.5 Flash-Lite (LLM Stats MRCR v2 8-needle) Gemma 4 E2B (LLM Stats MRCR v2 8-needle) Gemini 2.5 Pro Preview 06-05 (LLM Stats MRCR v2 8-needle) Gemma 3 27B (LLM Stats MRCR v2 8-needle) GPT-5 (Lean, ~42/660, pass@1, ~10-turn ReAct) Gemini 3 Pro (Challenge Avg@3) GPT-5.5 high (Challenge Avg@3) Qwen3 32B + SWE-agent (public, launch revision) GPT-4o + SWE-agent (public, launch revision) Claude Sonnet 4 + SWE-agent (public, launch revision) Claude Opus 4.1 + SWE-agent (public, launch revision) GPT-5 + SWE-agent (public, launch revision) Claude 4.5 Opus (high reasoning; mini-SWE-agent 2.0.0) Gemini 3 Flash (high reasoning; mini-SWE-agent 2.0.0) MiniMax M2.5 (high reasoning; mini-SWE-agent 2.0.0) Claude Opus 4.6 (mini-SWE-agent 2.0.0) GPT-5-2 Codex (mini-SWE-agent 2.0.0) GLM-5 (high reasoning; mini-SWE-agent 2.0.0) GPT-5-2 (high reasoning; mini-SWE-agent 2.0.0) Claude 4.5 Sonnet (high reasoning; mini-SWE-agent 2.0.0) Kimi K2.5 (high reasoning; mini-SWE-agent 2.0.0) DeepSeek V3.2 (high reasoning; mini-SWE-agent 2.0.0) Gemini 3 Pro (mini-SWE-agent 2.0.0) Claude 4.5 Haiku (high reasoning; mini-SWE-agent 2.0.0) GPT-5 Mini (mini-SWE-agent 2.0.0) GPT-4o (198-task Diamond offline) o1 (198-task Diamond offline) Claude Code + Fable 5 (xhigh; TB 2.1) Codex + GPT-5.5 (xhigh; TB 2.1) Terminus 2 + Fable 5 (high; TB 2.1) Cursor CLI + Grok 4.5 (high; TB 2.1) Claude Code + Opus 4.8 (high; TB 2.1) Codex + GPT-5.6 Terra (max; TB 2.1) Terminus 2 + GPT-5.5 (xhigh; TB 2.1) mini-SWE-agent + Muse Spark 1.1 (xhigh; TB 2.1) Codex + GPT-5.6 Luna (max; TB 2.1) Claude Code + Sonnet 5 (high; TB 2.1) Terminus 2 + Gemini 3 Pro (high; TB 2.1) Claude Code + Opus 4.7 (max; TB 2.1) Terminus 2 + Opus 4.7 (max; TB 2.1) Gemini CLI + Gemini 3 Pro (high; TB 2.1) Gemini CLI + Gemini 3.1 Pro (high; TB 2.1) Terminus 2 + Gemini 3.1 Pro (high; TB 2.1) Claude Code + GLM-5.1 (max; TB 2.1) GPT-4o (few-shot) GPT-4o (full benchmark) GPT-4 Turbo + SWE-agent Gemini 1.5 Pro (3-shot CoT) Llama 3 70B (7-shot) llama 3 70b-instruct llama 3 8b-instruct Llama 3 70B CodeQwen1.5-7B-Chat CodeQwen1.5-7B base (Mercury-eval Overall Beyond, 5 samples) Llama 3 70B (5-shot) GPT-4-Turbo (April 2024) GPT-4-Turbo Claude 3 Opus (3-shot CoT) Claude 3 Opus (BM25 retrieval) Claude 3 Opus (5-shot) StarCoder2-15B base (Mercury-eval Overall Beyond, 5 samples) GPT-4 (paper Table 3 Average) YaRN-Mistral (paper Table 3 Average) Kimi-Chat (paper Table 3 Average) Claude 2 (paper Table 3 Average) GPT-4V (full benchmark) UN codellama-70b-instruct Gemini Ultra 1.0 (3-shot CoT) Gemini Ultra (10-shot decontaminated) PaLM 540B (translate-to-English CoT) GPT-4-Turbo (November 2023) DeepSeek-Coder-33B base (Mercury-eval Overall Beyond, 5 samples) Claude 2 (BM25 retrieval) GPT-3.5-Turbo-16k (paper Table 3 OverAll) Llama2-7B-chat-4k (paper Table 3 OverAll) LongChat-v1.5-7B-32k (paper Table 3 OverAll) XGen-7B-8k (paper Table 3 OverAll) InternLM-7B-8k (paper Table 3 OverAll) ChatGLM2-6B (paper Table 3 OverAll) ChatGLM2-6B-32k (paper Table 3 OverAll) Vicuna-v1.5-7B-16k (paper Table 3 OverAll) UN codellama-34b-instruct UN codellama-13b-instruct CodeLlama-34B base (Mercury-eval Overall Beyond, 5 samples) Llama 2 70B (5-shot) T0pp (paper Table 3 Avg) Flan-T5 (paper Table 3 Avg) Flan-UL2 (paper Table 3 Avg) DaVinci003 (paper Table 3 Avg) ChatGPT (paper Table 3 Avg) Claude (paper Table 3 Avg) GPT-4 (paper Table 3 Avg) CPACE GPT-4 (EvalPlus Table 3, greedy, HumanEval base pass@1) GPT-4 (May 2023) Chameleon (GPT-4, Text-GT, tools) text-davinci-003 ChatGPT (gpt-3.5-turbo) GPT-4 GPT-4 (3-shot CoT) GPT-4 (5-shot CoT) GPT-4 base (10-shot) GPT-4 (full MATH) GPT-4 (5-shot) gpt-3.5-turbo LLaMA-65B SantaCoder-1.1B (MultiPL-HumanEval Python, pass@1) PaLM 540B Codex (code-davinci-002) Vega v2 GPT-3 + PromptPG (2-shot CoT, Text-GT) InCoder-6.7B (MultiPL-HumanEval Python, pass@1) code-davinci-002 (MultiPL-HumanEval Python, pass@1) PaLM 540B (5-shot) Chinchilla 70B PaLM 540B (CoT + self-consistency) ST-MoE-32B AlphaCode 9B (test set, 10@100k, no clustering) AlphaCode 41B (test set, 10@100k, no clustering) AlphaCode 41B + clustering (test set, 10@100k) PaLM 540B (8-shot CoT) Naive (paper Table 2 Avg) BART 256 (paper Table 2 Avg) BART 512 (paper Table 2 Avg) BART 1024 (paper Table 2 Avg) LED 1024 (paper Table 2 Avg) LED 4096 (paper Table 2 Avg) LED 16384 (paper Table 2 Avg) Gopher 280B KEAR ensemble KEAR single model GPT-3 175B (fine-tuned) ERNIE 3.0 Codex-12B (original paper, 164-task HumanEval pass@1 estimator) Random (paper Table 3 R-1) Ext. Oracle (paper Table 3 R-1) TextRank (paper Table 3 R-1) PGNet (paper Table 3 R-1) BART (paper Table 3 R-1) HMNet* (paper Table 3 R-1) PGNet (gold spans) (paper Table 3 R-1) BART (gold spans) (paper Table 3 R-1) HMNet (gold spans) (paper Table 3 R-1) GPT-3 175B (full MATH) Seq2Seq (CodeSearchNet summarization, six-language overall BLEU) Transformer (CodeSearchNet summarization, six-language overall BLEU) RoBERTa encoder (CodeSearchNet summarization, six-language overall BLEU) CodeBERT encoder (CodeSearchNet summarization, six-language overall BLEU) DeBERTa-xxlarge (1.5B) DeBERTa (TuringNLRv4) DeBERTa ensemble (TuringNLRv4) DeBERTa 1.5B (single model) GPT-3 175B GPT-3 175B (few-shot) ELECTRA-Large T5 (ensemble) T5-11B ALBERT (ensemble) RoBERTa-large RoBERTa ensemble RoBERTa BERT-Large (fine-tuned) RoBERTa-large (fine-tuned) XLNet (ensemble) BERT++ MT-DNN BERT-Large
Related Benchmark — Select Benchmark (63 Available) — QuALITY MBPP+ Mercury MGSM MMLU MMLU-Pro MMLU-Redux MT-Bench MultiPL-E Needle-in-a-Haystack OlympiadBench Omni-MATH OpenAI-MRCR PutnamBench QASPER QMSum MBPP RULER SCROLLS SimpleQA Soohak StructFlowBench SuperGLUE SWE-bench SWE-bench Pro SWE-bench Verified SWE-Lancer TabMWP Terminal-Bench TruthfulQA WinoGrande ZeroSCROLLS GLUE AGIEval AIME ARC-AGI ARC-AGI-2 ARC-AGI-3 BIG-Bench BIG-Bench Hard CodeContests CodeXGLUE CommonsenseQA ComplexBench CyberSecEval FireBench FrontierCode FrontierMath AgentIF GPQA GSM-Hard GSM8K HellaSwag HumanEval HumanEval+ Humanity's Last Exam IFEval InfiniteBench LiveCodeBench LongBench LongBench-Pro MATH MathArena
Related AI Lab — Select Lab / Company (19 Available) — 01.AI AI21 Labs Alibaba Cloud / Qwen Allen Institute for AI (AI2) Amazon AWS Anthropic Apple Baidu ByteDance Research Center for AI Safety Cohere Databricks DeepSeek Epoch AI Google DeepMind LG AI Research Meta AI Microsoft Research MiniMax Mistral AI Moonshot AI Ollama (Inference Runtime) OpenAI Perplexity AI Reka AI Scale AI Snowflake Stanford CRFM / HELM Tencent Tsinghua University UC Berkeley / LMSYS Upstage xAI Zhipu AI