OpenAIAPIo3-mini high (Epoch independent eval, Tiers 1–3, tools-on)ParamsUndisclosedContext128kSpeed—ReleaseFeb 2025
OpenAIAPIGPT-5 (Lean, ~42/660, pass@1, ~10-turn ReAct)ParamsUndisclosedContext128kSpeed—ReleaseJun 2024
OpenAIAPIGPT-4o + SWE-agent (public, launch revision)ParamsUndisclosedContext128kSpeed—ReleaseJun 2024
OpenAIAPIGPT-5 + SWE-agent (public, launch revision)ParamsUndisclosedContext128kSpeed—ReleaseJun 2024
OpenAIAPIGPT-5-2 (high reasoning; mini-SWE-agent 2.0.0)ParamsUndisclosedContext128kSpeed—ReleaseJun 2024
OpenAIAPIKimi K2.5 (high reasoning; mini-SWE-agent 2.0.0)ParamsUndisclosedContext128kSpeed—ReleaseJun 2024
OpenAIAPIStarCoder2-15B base (Mercury-eval Overall Beyond, 5 samples)ParamsUndisclosedContext128kSpeed—ReleaseFeb 2024
OpenAIAPILongChat-v1.5-7B-32k (paper Table 3 OverAll)ParamsUndisclosedContext128kSpeed—ReleaseAug 2023
OpenAIAPIGPT-4 (EvalPlus Table 3, greedy, HumanEval base pass@1)ParamsUndisclosedContext128kSpeed—ReleaseMay 2023
OpenAIAPISantaCoder-1.1B (MultiPL-HumanEval Python, pass@1)ParamsUndisclosedContext128kSpeed—ReleaseJan 2023
OpenAIAPIcode-davinci-002 (MultiPL-HumanEval Python, pass@1)ParamsUndisclosedContext128kSpeed—ReleaseAug 2022
OpenAIAPICodex-12B (original paper, 164-task HumanEval pass@1 estimator)ParamsUndisclosedContext128kSpeed—ReleaseJul 2021
OpenAIAPISeq2Seq (CodeSearchNet summarization, six-language overall BLEU)ParamsUndisclosedContext128kSpeed—ReleaseFeb 2021
OpenAIAPITransformer (CodeSearchNet summarization, six-language overall BLEU)ParamsUndisclosedContext128kSpeed—ReleaseFeb 2021