Coding agents
Measured on FrontierCode-AA 2.0, Terminal-Bench 4.1 and SWE-Atlas-QnA, with cost and time per task. Unmeasured IDEs are profiled without scores.
| Rank | Autonomous Agent / AI IDE | Underlying Model | Index | FrontierCode | Terminal-Bench | Cost / task | Pricing Tier | Action |
|---|---|---|---|---|---|---|---|---|
| #1 | Claude Codeby Anthropic • Autonomous CLI AgentAnthropic's high-agency CLI agent that navigates repositories, runs terminals, and commits verified PRs. | Claude Fable 5.2 (Xhigh, Default Fallback) | 58.5 | 49.2% | 64.1% | $15.47 | PaidPay-as-you-go via Anthropic API (Claude 3.7 Sonnet tokens). Built-in prompt caching reduces cost by up to 90%. | Details |
| #2 | Google Antigravityby Google • Agent-First IDEGoogle's agent-first development platform: an AI editor plus a manager view for running several coding agents in parallel. | Gemini 4 Carbon (High) | 55.5 | 38.0% | 64.6% | $7.13 | FreemiumFree for individuals with usage limits | Details |
| #3 | Muse Codeby Meta • Coding AgentCoding agent harness from Meta, measured on the Artificial Analysis Coding Agent Index. | Muse Wonder Full (Max) | 54.5 | 35.0% | 59.6% | — | PaidSee the vendor for current pricing. | Details |
| #4 | Codexby OpenAI • Coding AgentCoding agent harness from OpenAI, measured on the Artificial Analysis Coding Agent Index. | GPT-6 Astra (Max) | 53.1 | 42.6% | 57.6% | $7.63 | PaidSee the vendor for current pricing. | Details |
| #5 | Kimi Code CLIby Kimi • Coding AgentCoding agent harness from Kimi, measured on the Artificial Analysis Coding Agent Index. | Kimi K3.1 Pro | 48.6 | 22.0% | 67.2% | $2.94 | PaidSee the vendor for current pricing. | Details |
| #6 | Grok Buildby SpaceXAI • Coding AgentCoding agent harness from SpaceXAI, measured on the Artificial Analysis Coding Agent Index. | Grok 4.7 (Xhigh) | 41.6 | 29.7% | 32.3% | $15.35 | PaidSee the vendor for current pricing. | Details |
| #7 | Mistral Vibeby Mistral • Coding AgentCoding agent harness from Mistral, measured on the Artificial Analysis Coding Agent Index. | Mistral Large 4 (High) | 40.6 | 20.5% | 42.4% | $1.34 | PaidSee the vendor for current pricing. | Details |
| #8 | Opencodeby OpenCode • Coding AgentCoding agent harness from OpenCode, measured on the Artificial Analysis Coding Agent Index. | GLM-5.3 (Max) | 40.1 | 21.3% | 40.4% | $5.41 | PaidSee the vendor for current pricing. | Details |
| #9 | Cursorby Anysphere • AI-First IDEThe AI-first code editor built for blazing-fast multi-file engineering and predictive completions. | Claude 3.7 Sonnet / DeepSeek-V3 | — | — | — | — | FreemiumFree tier with 2,000 completions | Details |
| #10 | Windsurfby Codeium • AI-First IDEThe next-generation agentic IDE featuring Cascade bidirectional flow between editor and terminal. | Claude 3.7 Sonnet / Cascade Flow | — | — | — | — | FreemiumFree tier with unlimited autocomplete | Details |
| #11 | Devinby Cognition AI • Autonomous Cloud AgentThe world's first fully autonomous AI software engineer, operating in an isolated cloud VM. | Cognition Proprietary Ensemble (Claude 3.7 + o3-mini) | — | — | — | — | PaidTeam plan starts at $500/month (includes dedicated compute hours) | Details |
| #12 | GitHub Copilotby GitHub & Microsoft • In-IDE ExtensionThe world's most widely adopted AI developer tool, deeply embedded across Microsoft's development ecosystem. | GPT-4o / Claude 3.5 Sonnet / o3-mini | — | — | — | — | FreemiumFree tier with 2,000 completions/month | Details |
| #13 | Clineby Cline Community • In-IDE ExtensionThe leading open-source autonomous coding agent for VS Code with human-in-the-loop permission controls. | Claude 3.7 Sonnet / DeepSeek-R1 | — | — | — | — | Open Source100% Free & Open Source (Apache 2.0). Pay model providers directly via API key (BYOK). | Details |
| #14 | Roo Codeby Roo Vet & Community • In-IDE ExtensionAdvanced multi-mode AI coding agent for VS Code with specialized Architect, Code, and Debug personas. | Claude 3.7 Sonnet / DeepSeek-V3 | — | — | — | — | Open Source100% Free & Open Source (Apache 2.0). Connect directly to any model provider API. | Details |
| #15 | Aiderby Paul Gauthier • Autonomous CLI AgentThe premier open-source AI pair programmer in your terminal with git-native automated commits. | Claude 3.7 Sonnet + DeepSeek-R1 | — | — | — | — | Open Source100% Free & Open Source (Apache 2.0). Connects to Anthropic, OpenAI, DeepSeek, or local LLMs. | Details |
| #16 | Lovableby Lovable Labs • Full-Stack Web BuilderThe fastest full-stack AI software engineer for building and deploying production web apps from natural language. | Claude 3.7 Sonnet + GPT-4o | — | — | — | — | FreemiumFree tier with daily prompt credits | Details |
| #17 | v0 by Vercelby Vercel • Full-Stack Web BuilderVercel's generative UI and full-stack Next.js developer assistant for production React applications. | Claude 3.5 Sonnet / GPT-4o | — | — | — | — | FreemiumFree tier with 200 credits/month | Details |
| #18 | Bolt.newby StackBlitz • Full-Stack Web BuilderIn-browser full-stack AI development environment running Node.js and dev servers in WebContainers. | Claude 3.7 Sonnet / DeepSeek-V3 | — | — | — | — | FreemiumFree tier with daily tokens | Details |
| #19 | Replit Agentby Replit • Full-Stack Web BuilderAutonomous cloud application builder that creates, debugs, and hosts full-stack software from scratch. | Claude 3.5 Sonnet / Replit Ensemble | — | — | — | — | PaidIncluded with Replit Core ($25/mo or $20/mo billed annually) with checkpoints and cloud hosting. | Details |
| #20 | Augment Codeby Augment • In-IDE ExtensionEnterprise AI coding platform built for massive million-line codebases and complex multi-repo architectures. | Augment Code Intelligence Ensemble | — | — | — | — | PaidDeveloper Pro at $20/month | Details |
| #21 | Continueby Continue.dev • In-IDE ExtensionThe leading open-source AI code assistant for VS Code and JetBrains with full local model support. | Custom / Any (Claude 3.7, DeepSeek, Ollama) | — | — | — | — | Open Source100% Free & Open Source (Apache 2.0). Connect to any API or local LLM via Ollama. | Details |
| #22 | Traeby ByteDance • AI-First IDEAdaptive AI IDE by ByteDance featuring Builder Mode for project-wide autonomous engineering. | Claude 3.5 Sonnet / GPT-4o | — | — | — | — | 100% FreeCurrently completely Free during public release with complimentary Claude 3.5 Sonnet & GPT-4o usage. | Details |
| #23 | Supermavenby Anysphere (Cursor) • In-IDE ExtensionThe fastest code autocomplete engine with an unprecedented 1,000,000 token context window. | Babble (Proprietary 1M Context Model) | — | — | — | — | FreemiumFree tier with unlimited standard completions | Details |
| #24 | Gooseby Block (Square) • Autonomous CLI AgentBlock's open-source on-machine developer agent that automates complex engineering tasks via MCP. | Claude 3.7 Sonnet / Databricks DBRX | — | — | — | — | Open Source100% Free & Open Source (Apache 2.0). Connects to Anthropic, Databricks, OpenAI, or Ollama. | Details |
| #25 | Codyby Sourcegraph • In-IDE ExtensionSourcegraph's AI assistant powered by code graph indexing for multi-repo enterprise context. | Claude 3.5 Sonnet / GPT-4o | — | — | — | — | FreemiumFree tier with 200 completions/month | Details |
How Autonomous Coding Agents are Benchmarked
Unlike raw LLMs evaluated on single-turn token completion, autonomous agents are measured by multi-file action loops, tool calls, and test suite outcomes.
A composite of three benchmarks with equal weight: FrontierCode-AA 2.0 (software engineering tasks), Terminal-Bench 4.1 (agentic terminal use) and SWE-Atlas-QnA-AA (technical Q&A). Each score is the mean over three attempts per task, measured by Artificial Analysis.
Measures whether the agent can execute shell commands, run test runners (pytest, npm test), parse stack traces, and recover autonomously when compilation or linter errors occur.
Tracks token consumption and API pricing per solved task. High pass rates achieved through runaway multi-hundred-turn loops can cost $20+/task, whereas optimal agents resolve bugs for under $1.50.
Autonomous agents rely on high-reasoning foundation backends (Claude 3.7 Sonnet, GPT-4o, DeepSeek V3) paired with specialized scaffolding, custom prompts, and tree-search strategies.