Platform Index • Real-Time DynamicUSA & India Dual-Region Optimized

TrustTheBench Platform Sitemap & Directory Index

Comprehensive visual index of verified AI benchmark suites, foundation models, research labs, capability insights, and forensic evaluation datasets. All links are crawlable and refreshed hourly.

64+Audited Benchmarks
699+Evaluated Models
18Coding Agents & IDEs
52+AI Research Labs
6Capability Domains
100%Verified Provenance

Core Platform Hubs & Interactive Tools

8 Routes

Autonomous Coding Agents & AI IDEs

18 Agents

Cursor

AI-First IDE

The AI-first code editor built for blazing-fast multi-file engineering and predictive completions.

Free tier with 2,000 completions; Pro at $20/month with 500 fast requests; Business at $40/user/mo.

Windsurf

AI-First IDE

The next-generation agentic IDE featuring Cascade bidirectional flow between editor and terminal.

Free tier with unlimited autocomplete; Pro at $15/month; Ultimate at $60/user/mo.

Claude Code

Autonomous CLI Agent

Anthropic's high-agency CLI agent that navigates repositories, runs terminals, and commits verified PRs.

Pay-as-you-go via Anthropic API (Claude 3.7 Sonnet tokens). Built-in prompt caching reduces cost by up to 90%.

Devin

Autonomous Cloud Agent

The world's first fully autonomous AI software engineer, operating in an isolated cloud VM.

Team plan starts at $500/month (includes dedicated compute hours); Enterprise custom pricing.

GitHub Copilot

In-IDE Extension

The world's most widely adopted AI developer tool, deeply embedded across Microsoft's development ecosystem.

Free tier with 2,000 completions/month; Pro at $10/mo; Business at $19/user/mo; Enterprise at $39/user/mo.

Cline

In-IDE Extension

The leading open-source autonomous coding agent for VS Code with human-in-the-loop permission controls.

100% Free & Open Source (Apache 2.0). Pay model providers directly via API key (BYOK).

Roo Code

In-IDE Extension

Advanced multi-mode AI coding agent for VS Code with specialized Architect, Code, and Debug personas.

100% Free & Open Source (Apache 2.0). Connect directly to any model provider API.

Aider

Autonomous CLI Agent

The premier open-source AI pair programmer in your terminal with git-native automated commits.

100% Free & Open Source (Apache 2.0). Connects to Anthropic, OpenAI, DeepSeek, or local LLMs.

Lovable

Full-Stack Web Builder

The fastest full-stack AI software engineer for building and deploying production web apps from natural language.

Free tier with daily prompt credits; Starter at $20/mo; Launch at $50/mo; Scale at $100/mo.

v0 by Vercel

Full-Stack Web Builder

Vercel's generative UI and full-stack Next.js developer assistant for production React applications.

Free tier with 200 credits/month; Premium at $20/month with 5,000 credits; Team at $30/user/mo.

Bolt.new

Full-Stack Web Builder

In-browser full-stack AI development environment running Node.js and dev servers in WebContainers.

Free tier with daily tokens; Pro at $20/month with 10M tokens; Pro 50 at $50/month with 26M tokens.

Replit Agent

Full-Stack Web Builder

Autonomous cloud application builder that creates, debugs, and hosts full-stack software from scratch.

Included with Replit Core ($25/mo or $20/mo billed annually) with checkpoints and cloud hosting.

Augment Code

In-IDE Extension

Enterprise AI coding platform built for massive million-line codebases and complex multi-repo architectures.

Developer Pro at $20/month; Enterprise custom pricing for large organizations.

Continue

In-IDE Extension

The leading open-source AI code assistant for VS Code and JetBrains with full local model support.

100% Free & Open Source (Apache 2.0). Connect to any API or local LLM via Ollama.

Trae

AI-First IDE

Adaptive AI IDE by ByteDance featuring Builder Mode for project-wide autonomous engineering.

Currently completely Free during public release with complimentary Claude 3.5 Sonnet & GPT-4o usage.

Supermaven

In-IDE Extension

The fastest code autocomplete engine with an unprecedented 1,000,000 token context window.

Free tier with unlimited standard completions; Pro at $10/month with 1M token context Babble model.

Goose

Autonomous CLI Agent

Block's open-source on-machine developer agent that automates complex engineering tasks via MCP.

100% Free & Open Source (Apache 2.0). Connects to Anthropic, Databricks, OpenAI, or Ollama.

Cody

In-IDE Extension

Sourcegraph's AI assistant powered by code graph indexing for multi-repo enterprise context.

Free tier with 200 completions/month; Pro at $9/month; Enterprise at $19/user/mo.

Capability Domains & Research Insights

6 Articles

AI Benchmark Suites Directory

64 Suites

LiveBench (Continuous)

active

Monthly refreshed benchmark designed to prevent test-set contamination across reasoning, coding, mathematics, data analysis, and instruction following.

QuALITY

nearing-saturation

Multiple-choice long-document QA where writers and validators read the full article, with a hard subset designed to defeat skimming.

Mercury

deprecated

Python code-synthesis benchmark that rewards both functional correctness and runtime efficiency against real solution distributions.

QMSum

nearing-saturation

Query-focused meeting summarization over long transcripts from academic, product, and committee meetings.

QASPER

nearing-saturation

Scientific-paper QA over full NLP papers, with abstractive, extractive, yes/no, and unanswerable answers plus supporting evidence.

PutnamBench

active

Formal theorem-proving on Putnam problems — models write a complete Lean/Isabelle/Coq proof, machine-checked by the proof assistant's kernel.

OpenAI-MRCR

active

MRCR v2 8-needle long-context benchmark where a model must recover the requested instance from repeated similar requests in a synthetic conversation.

Omni-MATH

deprecated

A 4,428-problem Olympiad mathematics suite spanning more than 33 subdomains and ten annotated difficulty levels.

OlympiadBench

active

Bilingual Olympiad and Gaokao mathematics and physics problems, including diagrams, open answers, and proofs.

Needle-in-a-Haystack

saturated

Synthetic retrieval stress test that hides facts at different depths and context lengths, then asks the model to recover them.

MultiPL-E

deprecated

Parallel HumanEval and MBPP translations for comparing code generation across many programming languages.

MT-Bench

nearing-saturation

80 curated multi-turn questions across 8 skills, each a two-turn conversation scored by a strong LLM judge on a 1–10 scale.

MMLU-Redux

saturated

A manual error audit of MMLU: experts re-annotated thousands of questions and found ~6.5% are broken — the evidence base for MMLU's label-noise ceiling.

MMLU-Pro

nearing-saturation

MMLU rebuilt to fix its flaws: 10 options instead of 4, trivia and label noise filtered out, and reasoning-heavy questions that reward chain-of-thought.

MMLU

saturated

57-subject multiple-choice exam spanning STEM, humanities, and social sciences — the defining knowledge benchmark of the 2020–2024 era.

MGSM

nearing-saturation

A human-translated 250-problem GSM8K subset for comparing grade-school mathematical reasoning across ten languages.

MBPP+

nearing-saturation

A cleaned 378-task MBPP evaluation with roughly 35 times more tests for stricter Python functional correctness.

RULER

nearing-saturation

Configurable synthetic benchmark that expands needle retrieval into multi-needle, multi-hop tracing, and aggregation tasks.

SCROLLS

nearing-saturation

Seven-task long-sequence suite covering summarization, QA, and NLI over naturally long English texts.

SimpleQA

active

4,326 short fact-seeking questions chosen to stump GPT-4 — a hallucination test graded correct / incorrect / not-attempted.

Soohak

active

Fresh research-level mathematics authored by mathematicians, plus unanswerable prompts that reward justified refusal.

StructFlowBench

nearing-saturation

155 multi-turn dialogues scored on fine-grained constraints plus six inter-turn dialogue relationships (recall, refinement, expansion, follow-up, summary).

SuperGLUE

saturated

Eight English NLU tasks spanning question answering, inference, causal reasoning, word sense, and coreference.

SWE-bench

deprecated

Real GitHub issues from 12 Python repositories, solved by generating patches that must pass repository tests.

SWE-bench Pro

active

Long-horizon repository tasks across public, held-out, and commercial codebases, with a continuing public model leaderboard.

SWE-bench Verified

saturated

A 500-task, engineer-reviewed subset of SWE-bench that became the standard repository-level coding-agent test.

SWE-Lancer

deprecated

Current 198-task offline patch benchmark derived from paid Upwork work, with historical mixed implementation and manager-task variants.

TabMWP

saturated

Grade-level math word problems that require combining a question with structured, textual, or image-rendered tables.

Terminal-Bench

active

Eighty-nine realistic terminal tasks graded in containers; version 2.1 repairs 2.0's dependency, resource, and specification defects.

TruthfulQA

saturated

817 adversarial questions probing whether models repeat common human misconceptions — famous for finding that bigger models were often less truthful.

WinoGrande

saturated

Binary fill-in-the-blank commonsense problems, scaled from Winograd schemas and adversarially filtered to reduce dataset shortcuts.

ZeroSCROLLS

nearing-saturation

Zero-shot long-text suite derived from SCROLLS with test-only tasks and new aggregation-style long-context challenges.

GLUE

saturated

Nine-task English NLU suite combining acceptability, sentiment, similarity, paraphrase, inference and coreference into one score.

AGIEval

saturated

Standardized human exams — SATs, LSATs, China's Gaokao and civil-service tests — repurposed to grade models against real human test-takers.

AIME

nearing-saturation

The MAA's competition exam — 15 integer-answer problems per test, refreshed yearly so each new exam is briefly contamination-free.

ARC-AGI

saturated

Colored-grid abstraction tasks that test few-shot rule induction, not memorized facts, language fluency, or school math.

ARC-AGI-2

nearing-saturation

The harder static ARC successor: human-calibrated colored-grid abstraction tasks, now pressured by 2026 frontier systems.

ARC-AGI-3

active

Interactive ARC environments that score agents by human-relative action efficiency, planning, exploration, and adaptation.

BIG-Bench

deprecated

204-task community-built mega-suite spanning reasoning, knowledge, math, code and social bias — the sprawling proving ground BBH was distilled from.

BIG-Bench Hard

saturated

23 BIG-Bench tasks where models trailed humans — the benchmark that proved chain-of-thought prompting works, then fell to the reasoning-model era.

CodeContests

deprecated

DeepMind's competitive-programming corpus with temporally split problems, human submissions, and generated correctness tests.

CodeXGLUE

deprecated

A broad code-intelligence suite spanning understanding, retrieval, completion, translation, repair, generation, and summarization.

CommonsenseQA

saturated

Five-choice questions derived from ConceptNet relations, designed to require everyday knowledge beyond a supplied passage.

ComplexBench

nearing-saturation

Bilingual benchmark of how models follow instructions when several constraints must hold at once — where stacking constraints is not linear.

CyberSecEval

deprecated

Meta's evolving suite for insecure code, cyberattack assistance, prompt injection, exploitation, SOC analysis, and autopatching.

FireBench

active

Evaluates instruction following for enterprise/API pipelines — exact output format, strict step ordering, ranking, and calibrated refusal across six categories.

FrontierCode

active

Private maintainer-authored repository tasks graded for mergeability: correctness, tests, scope discipline, style, and code quality.

FrontierMath

active

Original, unpublished research-level math problems by expert mathematicians — mostly private to resist contamination. Frontier models solved under 2% at launch.

AgentIF

nearing-saturation

707 instructions from 50 real-world agents, averaging 1,723 words and 11.9 constraints — whether instruction following survives deployment-scale prompts.

GPQA

nearing-saturation

PhD-written science questions so hard that skilled non-experts with Google score 34% — the 'Google-proof' exam, reported on its 198-question Diamond subset.

GSM-Hard

deprecated

GSM8K test problems with unusually large numerical values, designed to separate reasoning from easy arithmetic.

GSM8K

saturated

8.5K grade-school math word problems (2–8 arithmetic steps) — the default elementary-math benchmark and the classic chain-of-thought demonstration.

HellaSwag

saturated

Choose the plausible continuation of an everyday event from one real ending and three adversarially filtered machine generations.

HumanEval

saturated

OpenAI's 164 hand-written Python function-synthesis tasks, graded by executing generated completions against unit tests.

HumanEval+

saturated

HumanEval's 164 Python tasks re-evaluated with roughly 80 times more tests to expose plausible but incorrect programs.

Humanity's Last Exam

active

~2,500 expert-written questions across 100+ subjects, each filtered to stump frontier models — designed to be the final closed-ended academic benchmark.

IFEval

deprecated

~500 prompts, each carrying an explicit, machine-checkable constraint (format, content, or style) that a response must respect.

InfiniteBench

nearing-saturation

100k+ token benchmark mixing realistic and synthetic tasks across English, Chinese, code, math, and retrieval.

LiveCodeBench

saturated

Continuously updated contest problems for time-windowed code generation, execution, test prediction, and self-repair evaluation.

LongBench

nearing-saturation

Bilingual multitask suite for long-context understanding across QA, summarization, few-shot learning, synthetic tasks, and code completion.

LongBench-Pro

active

Realistic bilingual long-context benchmark with 1,500 natural samples across 11 primary and 25 secondary tasks from 8k to 256k tokens.

MATH

saturated

12,500 AMC/AIME-level competition problems with worked solutions — usually evaluated on its 500-problem MATH-500 subset, the reasoning-model math standard.

MathArena

active

A continuously refreshed platform evaluating models on newly released math competitions, proofs, research problems, and formalization.

MBPP

saturated

Mostly Basic Programming Problems: 974 crowd-sourced Python synthesis tasks intended for entry-level programmers.

Evaluated AI Foundation Models & LLMs

699 Models

GPT-6.1 Sol Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 5.5

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 4 Argon

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Fable 5

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Sol

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Muse Wonder Full

Meta AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Fable 5.1

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 5

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Sonnet 5.5

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-6 Astra

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-6 Luna

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.5 Thinking

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.8 Flash

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V4.1 Flash

DeepSeek

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Kimi K3

Moonshot AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Grok 4

xAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Grok 3

xAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.7 Flash

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen 3.8 Max Prime

Alibaba Cloud / Qwen

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Grok 4.6

xAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Sonnet 5

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Muse Wonder

Meta AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3 Pro

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 3.7 Sonnet

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Hy4

Tencent

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

OpenAI Decisions API

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V4-Pro

DeepSeek

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GLM 5.3 Flash

Zhipu AI (Z.ai)

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o3

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.6 Flash

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.8 Max Prime

Alibaba Cloud / Qwen

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 5.5

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-6 Astra Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 5

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Terra Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Sol Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3-235B

Alibaba Cloud / Qwen

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 4.8

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3 Pro Image (Nano Banana Pro)

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.5 Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 4.7

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.4 Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.1 Pro Custom Tools

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.1 Pro

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 4.6

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.2 Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 4.5

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3 Pro Image (Nano Banana Pro)

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5 Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 4.1 Opus

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o3 Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Perplexity: Sonar Pro Search

Perplexity AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Perplexity: Sonar Pro

Perplexity AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o1-pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Nemotron 3 Ultra 550B

NVIDIA AI & NIM

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 2.5 Pro

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Space Bunny Alpha

Stealth AI Research

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek-R1

DeepSeek

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

QwQ Plus

Alibaba Cloud / Qwen

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Amazon Nova Premier 1.0

Amazon AWS Bedrock

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GLM-5.2

Zhipu AI (Z.ai)

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 4 Maverick

Meta AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Sonar Reasoning Pro

Perplexity AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Ollama DeepSeek-R1 (Q4_K_M)

Ollama (Inference Runtime)

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Luna

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Jev 1.14

TypeSafe AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax M3

MiniMax

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o3-mini

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 4 Scout

Meta AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V4 Flash 0731

DeepSeek

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-6.1 Terra Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MoonshotAI: Kimi K3.1 Pro

Moonshot AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-6.1 Sol Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-6 Sol Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 5.3 Prime

Zhipu AI (Z.ai)

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Xiaomi: MiMo-V2.6-Pro-UltraSpeed

Xiaomi

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.8 Max (0902)

Alibaba Cloud / Qwen

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.7 Max

Alibaba Cloud / Qwen

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V4 Pro 0423

DeepSeek

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.1-Codex-Max

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Sarvam 105B

Sarvam AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 2.5 Pro

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 2.5 Pro 06-05

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Perplexity: Sonar Reasoning Pro

Perplexity AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Amazon: Nova Pro 1.0

Amazon AWS Bedrock

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Jev

TypeSafe AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Large

Mistral AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V4-Flash

DeepSeek

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 2.0 Flash Thinking

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Nemotron 3 Ultra

NVIDIA AI & NIM

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.5 Flash-Lite

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o1

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4.5

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Microsoft-Decision-1

Microsoft Research & Azure

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 2.0 Pro

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiMo-V2.6-Flash

Xiaomi

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Krutrim-2

Krutrim AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

QwQ 32B

Alibaba Cloud / Qwen

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Large 3

Mistral AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen 2.5 Max

Alibaba Cloud / Qwen

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama-3.1-Nemotron-70B-Instruct

NVIDIA AI & NIM

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Amazon Nova Pro 1.0

Amazon AWS Bedrock

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen 2.5 72B Instruct

Alibaba Cloud / Qwen

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiMo-V2.5

Xiaomi

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V4 Pro 0813

DeepSeek

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Luna Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Luna

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Nemotron 3 Ultra

NVIDIA AI & NIM

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax: MiniMax M3

MiniMax

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Xiaomi: MiMo-V2.5-Pro

Xiaomi

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax: MiniMax M2.7

MiniMax

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax: MiniMax M2-her

MiniMax

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax: MiniMax M2.1

MiniMax

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax: MiniMax M2

MiniMax

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax: MiniMax M1

MiniMax

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Nemotron 3 Super 120B

NVIDIA AI & NIM

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax: MiniMax-01

MiniMax

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Large 4

Mistral AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Large 3 2512

Mistral AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Phi-4

Microsoft Research & Azure

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 3.5 Sonnet

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 3.3 70B Instruct

Meta AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Hunyuan-Large

Tencent

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Nemotron-4-340B-Instruct

NVIDIA AI & NIM

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 3 27B

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Sarvam 30B

Sarvam AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Large 2

Mistral AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Kimi k1.5

Moonshot AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Sonar Pro

Perplexity AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Ollama Llama 3.3 (Q4_K_M)

Ollama (Inference Runtime)

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Yi-Lightning

01.AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Krutrim Pro

Krutrim AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Amazon Nova Pro

Amazon AWS Bedrock

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek-V3

DeepSeek

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 2.0 Flash

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Amazon Nova 2 Lite

Amazon AWS Bedrock

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax-Text-01

MiniMax

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 3.1 405B

Meta AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

WizardLM-2 8x22B

Microsoft Research & Azure

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen 2.5 Coder 32B

Alibaba Cloud / Qwen

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ERNIE 4.0 Turbo

Baidu

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

BharatGPT-14B

CoRover / BharatGPT

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax: MiniMax M3.1 Flash

MiniMax

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-6 Luna Pro

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-6 Luna

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Xiaomi: MiMo-V2.6-Pro

Xiaomi

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax: MiniMax M2.5

MiniMax

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Upstage: Solar Pro 4

Upstage

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Sonar Large

Perplexity AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GLM-4-Plus

Zhipu AI (Z.ai)

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 3.3 70B

Meta AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Phi-4-multimodal

Microsoft Research & Azure

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Reka Core

Reka AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Large 2

Mistral AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Hanooman 7B

Hanooman AI (SML)

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Codestral 25.01

Mistral AI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 2 27B

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Solar Pro

Upstage

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 3 Opus

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 1.5 Pro

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Lyria 3 Pro

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DBRX Instruct

Databricks

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 3.5 Haiku

Anthropic

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Snowflake Arctic

Snowflake

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Amazon Nova Lite

Amazon AWS Bedrock

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Airavata 7B

AI4Bharat

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Command R+

Cohere

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o mini

OpenAI

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 1.5 Flash

Google DeepMind

Flagship frontier model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Muse Wonder

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-6 Astra

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Fable 5.1

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MoonshotAI: Kimi K3

Moonshot AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Terra

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Sol

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Fable 5

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT Chat Latest

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.5

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.4 Image 2

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.4

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.3-Codex

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Sonnet 4.6

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.2-Codex

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.2 Chat

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.2

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Sonnet 4.5

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Amazon: Nova Premier 1.0

Amazon AWS Bedrock

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o1

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4 Turbo

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Haiku 5.5

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

QwQ-32B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-6.1 Terra

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Muse Spark Open

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 4 Argon

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Nano Banana 2.1

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-6.1 Sol

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Sonnet 5.5

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-6 Sol

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

SpaceXAI: Grok 4.7

xAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.8 Flash

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Muse Spark 1.3

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.7 Flash

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.8 2.4T A95B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

SpaceXAI: Grok 4.6

xAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Muse Spark 1.2

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.6 Flash

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Muse Spark 1.1

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

SpaceXAI: Grok 4.5

xAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Sonnet 5

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 5.2

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MoonshotAI: Kimi K2.7 Code

Moonshot AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.5 Flash

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Medium 3.5

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MoonshotAI: Kimi K2.6

Moonshot AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 5.1

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 5V Turbo

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.4 Mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 5 Turbo

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.5 397B A17B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 5

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT Audio

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.1

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.1-Codex

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5 Image

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Haiku 4.5

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 235B A22B Thinking 2507

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o4 Mini High

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o3

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o4 Mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Perplexity: Sonar Deep Research

Perplexity AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o3 Mini High

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o3 Mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mixtral 8x22B Instruct

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-3.5 Turbo 16k

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4.1

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 2.5 Flash (Thinking)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 3.5 Sonnet (v2)

Anthropic

flagship tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Nemotron-4-340B-Reward

NVIDIA AI & NIM

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Grok 3 Mini

xAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 5.3 FlashX

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Hy4

Tencent

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 5.3

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.8 27B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Muse Glimmer 30B

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ByteDance Seed: Seed 2.1 Turbo

ByteDance Research

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ByteDance Seed: Seed-2.0-Code

ByteDance Research

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.5 Flash Lite

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.1 Flash Lite Image

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.7 Plus

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.1 Flash Image

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

SpaceXAI: Grok Build 0.1

xAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.1 Flash Lite

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

SpaceXAI: Grok 4.3

xAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.6 Flash

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.6 27B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Xiaomi: MiMo-V2.5

Xiaomi

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.5 Plus 2026-04-20

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.6 Plus

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.4 Nano

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

SpaceXAI: Grok 4.20 Multi-Agent

xAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

SpaceXAI: Grok 4.20

xAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ByteDance Seed: Seed-2.0-Lite

ByteDance Research

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.1 Flash Lite

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.1 Flash Image

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.5-27B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.5-122B-A10B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.5 Plus 2026-02-15

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MoonshotAI: Kimi K2.5

Moonshot AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT Audio Mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3 Flash

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Devstral 2 2512

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.1-Codex-Mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MoonshotAI: Kimi K2 Thinking

Moonshot AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5 Image Mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 VL 235B A22B Instruct

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 Next 80B A3B Instruct

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MoonshotAI: Kimi K2 0905

Moonshot AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Medium 3.1

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5 Mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MoonshotAI: Kimi K2 0711

Moonshot AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ByteDance Seed: Seed 1.6

ByteDance Research

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Medium 3

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 2.5 Flash Image

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 2.5 Flash

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Command A+

Cohere

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

R1 0528

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V3 0324

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V3

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 4.7

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 4.6

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 4.5V

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 4.5

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-3.5 Turbo (older v0613)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4.1 Mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Nemotron 3 Super

NVIDIA AI & NIM

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-3.5 Turbo Instruct

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-3.5 Turbo

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Sarvam Vision (Document AI)

Sarvam AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Saaras v3 (Speech-to-Text)

Sarvam AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V4.1 Flash 1017

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Nemotron 3.5 Super VL

NVIDIA AI & NIM

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

LiquidAI: LFM2.5-24B-A2B

Liquid AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Haiku 5.5

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Xiaomi: MiMo-V2.6-Flash

Xiaomi

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.8 Omni Flash

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V4.1 Flash

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.8 Flash

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V4 Flash Vision Exp

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

LiquidAI: LFM2.5-2.6B

Liquid AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V4 Flash 0731

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Hy3

Tencent

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Hy-MT2-30B-A3B

Tencent

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Hy-MT2-7B

Tencent

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Hy3

Tencent

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.6 35B A3B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 4 26B A4B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 4 31B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.5-9B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.5-35B-A3B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.5-Flash

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ByteDance Seed: Seed-2.0-Mini

ByteDance Research

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Nemotron 3 Nano 30B A3B

NVIDIA AI & NIM

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Voxtral Small 24B 2507

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

gpt-oss-safeguard-20b

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 VL 8B Instruct

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 VL 30B A3B Thinking

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 VL 30B A3B Instruct

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 Coder Flash

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5 Nano

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

gpt-oss-120b

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 Coder 30B A3B Instruct

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 30B A3B Instruct 2507

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 Coder 480B A35B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 235B A22B Instruct 2507

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Hunyuan A13B Instruct

Tencent

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ByteDance Seed: Seed 1.6 Flash

ByteDance Research

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 30B A3B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 14B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 32B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 2.5 Flash Lite

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 4 Maverick

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 4 Scout

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 3 27B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Saba

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3 Coder Next

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen2.5 VL 72B Instruct

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen-Plus

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V3.2

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V3.2 Exp

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V3.1 Terminus

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V3.1

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 3.3 70B Instruct

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Amazon: Nova Lite 1.0

Amazon AWS Bedrock

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Sonar

Perplexity AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Perplexity: Sonar

Perplexity AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen2.5 Coder 32B Instruct

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 3.2 1B Instruct

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 3.2 3B Instruct

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen2.5 72B Instruct

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Command R (08-2024)

Cohere

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 3.1 70B Instruct

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o-mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o-mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 2 27B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Codestral 2508

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Small 4

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Small 3.1 24B

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 4.7 Flash

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 4.6V

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 4.5 Air

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4.1 Nano

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Bulbul v3 (Text-to-Speech)

Sarvam AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Amazon Nova 2 Lite

Amazon AWS Bedrock

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Amazon Nova Lite 1.0

Amazon AWS Bedrock

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen 3.8 Max Prime

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

IndicTrans-2

AI4Bharat

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

TypeSafe: Jev Router

TypeSafe AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Upstage: Solar Mini 4

Upstage

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Switchyard

NVIDIA AI & NIM

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Muse Spark 1.3 Contributor

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Z.ai: GLM 5.3 Flash

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Nemotron 3.5 Lightning

NVIDIA AI & NIM

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Muse Spark 1.2 Contributor

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen3.7 Flash

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Nemotron 3.5 Content Safety

NVIDIA AI & NIM

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Hy-MT2-1.8B

Tencent

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V4 Flash 0423

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Lyria 3 Clip

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Reka Edge

Reka AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Ministral 3 14B 2512

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

gpt-oss-20b

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ByteDance: UI-TARS 7B

ByteDance Research

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama Guard 4 12B

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 3 4B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 3 12B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Reka Flash 3

Reka AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Small 3.2 24B

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Small 3

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Phi 4

Microsoft Research & Azure

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Command R7B (12-2024)

Cohere

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Amazon: Nova Micro 1.0

Amazon AWS Bedrock

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Ministral 3 8B 2512

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Ministral 3 3B 2512

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen: Qwen2.5 7B Instruct

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 3.1 8B Instruct

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Mistral Nemo

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Sarvam 2B (Sarvam-1)

Sarvam AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Phi-4-mini

Microsoft Research & Azure

small tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Amazon Nova Micro 1.0

Amazon AWS Bedrock

small tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 2 9B

Google DeepMind

small tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 2 2B

Google DeepMind

small tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen2.5-14B-Instruct-1M

Alibaba Cloud / Qwen

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3-14B

Alibaba Cloud / Qwen

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3-32B

Alibaba Cloud / Qwen

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen2.5-7B-Instruct-1M

Alibaba Cloud / Qwen

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Jamba-1.5-mini

AI21 Labs

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

EXAONE-4.0-32B

LG AI Research

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Terra

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.4 mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.4 nano

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5 + SWE-agent

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5-2 Codex

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5-2

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5 Mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Codex + GPT-5.6 Terra (max; TB 2.1)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Terra

OpenAI

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Kimi K2.6

OpenAI

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Mythos

Anthropic

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 4.7

Anthropic

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Muse Spark 1.1

Meta AI

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3.7 Max

Alibaba Cloud / Qwen

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GLM-5.1

Zhipu AI (Z.ai)

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax M3

MiniMax

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 4.6

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 4 31B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.1 Flash-Lite

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 4 26B-A4B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 4 12B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.1 Pro

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 4 E4B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3 Flash

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 4 E2B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.6 Flash

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.5 Flash-Lite

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.6 Flash

Google DeepMind

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Luna

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Codex + GPT-5.6 Luna (max; TB 2.1)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.5 Pro (High)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Sol (xHigh)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.5 (xHigh)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Sol (Max)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.5 (High)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.5

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.5 (xhigh, expected performance)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6-Sol (max, expected performance)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Sol

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.5

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.5 high

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Codex + GPT-5.5 (xhigh; TB 2.1)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Terminus 2 + GPT-5.5 (xhigh; TB 2.1)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Luna

OpenAI

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.6 Sol

OpenAI

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-5.5

OpenAI

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Code + Opus 4.8 (high; TB 2.1)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Code + Sonnet 5 (high; TB 2.1)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

mini-SWE-agent + Muse Spark 1.1 (xhigh; TB 2.1)

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

SWE-1.7

Cognition AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Sonnet 5

Anthropic

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3-235B-A22B-Thinking-2507

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3-235B-A22B

Alibaba Cloud / Qwen

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Cursor CLI + Grok 4.5 (high; TB 2.1)

xAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Grok 4.5

xAI

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GLM-5.2

Zhipu AI (Z.ai)

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Kimi K2.7 Code

Moonshot AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Fable 5

Anthropic

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Code + Fable 5 (xhigh; TB 2.1)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Terminus 2 + Fable 5 (high; TB 2.1)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 4.8 (High)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 4.8

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 4.8

Anthropic

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3.7 Plus (qwen3-7-plus; closed)

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3.7 Max (qwen3-7-max; closed)

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.5 Flash

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3 Pro

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3 Pro

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Terminus 2 + Gemini 3 Pro (high; TB 2.1)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini CLI + Gemini 3 Pro (high; TB 2.1)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3 Pro

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini CLI + Gemini 3.1 Pro (high; TB 2.1)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Terminus 2 + Gemini 3.1 Pro (high; TB 2.1)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Code + Opus 4.7 (max; TB 2.1)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Terminus 2 + Opus 4.7 (max; TB 2.1)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Code + GLM-5.1 (max; TB 2.1)

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3.6-27B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 4.7 (High)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 4.7 (Max)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3.6-35B-A3B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

SWE-1.6

Cognition AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3.1 Pro

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Kimi K2.5

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 4.5 Opus

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 4.6

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 4.5 Sonnet

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 4.5 Haiku

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 3 Flash

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GLM-5

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax M2.5

MiniMax

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude-4-Sonnet

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3-Next-80B-A3B-Thinking

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3-30B-A3B-Thinking-2507

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

MiniMax-M2

MiniMax

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Anthropic Opus 4.6 (Max)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

NVARC

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Sonnet 4 + SWE-agent

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 4.1 + SWE-agent

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen3 32B + SWE-agent

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude Opus 4

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 4 Sonnet

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini-2.5-Flash

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o4-mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemma 3 27B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o3

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o3-low

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini-2.5-Pro

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 2.5 Pro 06-05

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ARChitects (Kaggle 2024 winner)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek-R1

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4.5

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Goedel-Prover

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

OpenAI o3-mini (High)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o3-mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek-R1-0528

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek-R1

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek-V3

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek-V3

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V3

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek-V3

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek V3.2

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Jeremy Berman system

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o1

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o1

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o1 (198-task Diamond offline)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen 2.5 Coder 32B Instruct

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen 2.5 Coder 32B Instruct

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen2.5-Coder 32B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

best frontier model

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Grok Beta

xAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 3.5 Sonnet

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 3.5 Sonnet

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 3.5 Sonnet

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 3.5 Sonnet

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Qwen2.5 72B

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o1-mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o1

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o1

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

o1-mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

OpenAI o1

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

O1 Mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

OpenAI o1-mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek-V2.5

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Jamba-1.5-large

AI21 Labs

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 3.1 405B

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o Mini

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek-Coder-V2-Instruct

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek-Coder-V2-Instruct

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o (few-shot)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o (full benchmark)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o + SWE-agent

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4o (198-task Diamond offline)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 3 70B

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

llama 3 70b-instruct

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

UN codellama-34b-instruct

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

llama 3 8b-instruct

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 3 70B

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

CodeLlama-34B base (Mercury-eval Overall Beyond, 5 samples)

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 3 70B

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

CodeQwen1.5-7B-Chat

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

CodeQwen1.5-7B base (Mercury-eval Overall Beyond, 5 samples)

Alibaba Cloud / Qwen

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4 Turbo + SWE-agent

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4-Turbo (April 2024)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4-Turbo

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4-Turbo

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 3 Opus

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 3 Opus (BM25 retrieval)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 3 Opus

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

StarCoder2-15B base (Mercury-eval Overall Beyond, 5 samples)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Kimi-Chat (paper Table 3 Average)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

YaRN-Mistral (paper Table 3 Average)

Mistral AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 1.5 Pro

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini 1.5 Pro 002

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini-1.5-pro

Google DeepMind

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

UN codellama-70b-instruct

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GLM-4.7 (glm-4-7; open weight)

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini Ultra 1.0

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gemini Ultra

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

PaLM 540B (translate-to-English CoT)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DeepSeek-Coder-33B base (Mercury-eval Overall Beyond, 5 samples)

DeepSeek

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4 Turbo (1106)

OpenAI

snapshot tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

LongChat-v1.5-7B-32k (paper Table 3 OverAll)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

XGen-7B-8k (paper Table 3 OverAll)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

InternLM-7B-8k (paper Table 3 OverAll)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Vicuna-v1.5-7B-16k (paper Table 3 OverAll)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama2-7B-chat-4k (paper Table 3 OverAll)

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ChatGLM2-6B (paper Table 3 OverAll)

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ChatGLM2-6B-32k (paper Table 3 OverAll)

Zhipu AI (Z.ai)

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

UN codellama-13b-instruct

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Llama 2 70B

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 2 (paper Table 3 Average)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude 2 (BM25 retrieval)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

T0pp (paper Table 3 Avg)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

DaVinci003 (paper Table 3 Avg)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ChatGPT (paper Table 3 Avg)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Claude (paper Table 3 Avg)

Anthropic

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Flan-T5 (paper Table 3 Avg)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Flan-UL2 (paper Table 3 Avg)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

CPACE

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

text-davinci-003

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4.1

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4 (paper Table 3 Average)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4V (full benchmark)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4 (paper Table 3 Avg)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4 (May 2023)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Chameleon (GPT-4, Text-GT, tools)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4 base

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-4 (full MATH)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-3.5-Turbo-16k (paper Table 3 OverAll)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ChatGPT (gpt-3.5-turbo)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

gpt-3.5-turbo

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

LLaMA-65B

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

SantaCoder-1.1B (MultiPL-HumanEval Python, pass@1)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Codex (code-davinci-002)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

PaLM 540B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Vega v2

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-3 + PromptPG

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

code-davinci-002 (MultiPL-HumanEval Python, pass@1)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

InCoder-6.7B (MultiPL-HumanEval Python, pass@1)

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

PaLM 540B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Chinchilla 70B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

PaLM 540B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ST-MoE-32B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

AlphaCode 9B (test set, 10@100k, no clustering)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

AlphaCode 41B (test set, 10@100k, no clustering)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

AlphaCode 41B + clustering (test set, 10@100k)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

PaLM 540B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Naive (paper Table 2 Avg)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

LED 1024 (paper Table 2 Avg)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

LED 4096 (paper Table 2 Avg)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

LED 16384 (paper Table 2 Avg)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

BART 256 (paper Table 2 Avg)

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

BART 512 (paper Table 2 Avg)

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

BART 1024 (paper Table 2 Avg)

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Gopher 280B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-3 175B (fine-tuned)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ERNIE 3.0

Baidu

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Codex-12B (original paper, 164-task HumanEval pass@1 estimator)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Random (paper Table 3 R-1)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Ext. Oracle (paper Table 3 R-1)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

TextRank (paper Table 3 R-1)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

PGNet (paper Table 3 R-1)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

PGNet (gold spans) (paper Table 3 R-1)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

BART (paper Table 3 R-1)

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

BART (gold spans) (paper Table 3 R-1)

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-3 175B (full MATH)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Seq2Seq (CodeSearchNet summarization, six-language overall BLEU)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

Transformer (CodeSearchNet summarization, six-language overall BLEU)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

RoBERTa encoder (CodeSearchNet summarization, six-language overall BLEU)

Meta AI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-3 175B

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

GPT-3 175B (few-shot)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ELECTRA-Large

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

T5 (ensemble)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

T5-11B

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

ALBERT (ensemble)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

RoBERTa-large

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

RoBERTa ensemble

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

RoBERTa

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

RoBERTa-large (fine-tuned)

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

BERT-Large (fine-tuned)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

XLNet (ensemble)

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

BERT++

OpenAI

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

BERT-Large

Google DeepMind

standard tier foundation model evaluated across LiveBench, SWE-bench, and reasoning benchmarks.

AI Research Labs & Model Creators

52 Labs

01.AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for 01.AI.

AI21 Labs

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for AI21 Labs.

AI4Bharat

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for AI4Bharat.

Alibaba Cloud / Qwen

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Alibaba Cloud / Qwen.

Allen Institute for AI (AI2)

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Allen Institute for AI (AI2).

Amazon AWS Bedrock

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Amazon AWS Bedrock.

Anthropic

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Anthropic.

Anysphere (Cursor)

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Anysphere (Cursor).

Apple

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Apple.

Augment Code

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Augment Code.

Baidu

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Baidu.

ByteDance Research

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for ByteDance Research.

Center for AI Safety

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Center for AI Safety.

Codeium (Windsurf)

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Codeium (Windsurf).

Cognition AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Cognition AI.

Cohere

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Cohere.

CoRover / BharatGPT

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for CoRover / BharatGPT.

Databricks

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Databricks.

DeepSeek

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for DeepSeek.

Epoch AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Epoch AI.

Google DeepMind

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Google DeepMind.

Hanooman AI (SML)

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Hanooman AI (SML).

Krutrim AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Krutrim AI.

LG AI Research

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for LG AI Research.

Liquid AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Liquid AI.

Meta AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Meta AI.

Microsoft Research & Azure

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Microsoft Research & Azure.

MiniMax

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for MiniMax.

Mistral AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Mistral AI.

Moonshot AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Moonshot AI.

NVIDIA AI & NIM

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for NVIDIA AI & NIM.

Ollama (Inference Runtime)

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Ollama (Inference Runtime).

OpenAI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for OpenAI.

Perplexity AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Perplexity AI.

Reka AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Reka AI.

Replit

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Replit.

Sarvam AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Sarvam AI.

Scale AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Scale AI.

Snowflake

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Snowflake.

Sourcegraph

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Sourcegraph.

StackBlitz (Bolt.new)

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for StackBlitz (Bolt.new).

Stanford CRFM / HELM

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Stanford CRFM / HELM.

Stealth AI Research

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Stealth AI Research.

Tencent

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Tencent.

Tsinghua University

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Tsinghua University.

TypeSafe AI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for TypeSafe AI.

UC Berkeley / LMSYS

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for UC Berkeley / LMSYS.

Upstage

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Upstage.

Vercel

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Vercel.

xAI

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for xAI.

Xiaomi

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Xiaomi.

Zhipu AI (Z.ai)

Lab Profile

Frontier AI research, evaluated model portfolio, and benchmark score provenance for Zhipu AI (Z.ai).

Forensic Governance, Feeds & Legal Notices

9 Protocols