benchmark
.
space
benchmarks
rankings
compare
voices
transcripts
papers
articles
DeepSWE leaderboard
DeepSWE
22 models tested · Updated 2026-07-09 · Verified sources only
GPT-5.6 Sol
leads at
73.0%
1
GPT-5.6 Sol
OpenAI ·
Blog/OpenAI
· 2026-07-09
Score as cited in Kimi K3 blog (source: OpenAI official).
73.0%
2
GPT-5.6 Sol
OpenAI ·
Blog/OpenAI
· 2026-07-09
DeepSWE v1.1 SOTA, edging Fable 5 (69.7%) and GPT-5.5 (67%).
72.7%
3
Claude Fable 5
Anthropic ·
Blog/Anthropic
· 2026-06-09
Score as cited in Kimi K3 blog (source: Anthropic official). Fable 5 hit fallbacks on 35% of tasks.
70.0%
4
GPT-5.6 Terra
OpenAI ·
Blog/OpenAI
· 2026-07-09
Terra matches Fable 5 (69.7%) on DeepSWE at lower cost.
69.6%
5
Kimi K3
Moonshot AI ·
DeepSWE Leaderboard/Datacurve
· 2026-07-16
Independent run on official DeepSWE leaderboard (mini-swe-agent harness, v1.1, ±5%). 4th overall, ahead of GPT-5.6 Luna and GPT-5.5.
69.0%
6
Kimi K3
Moonshot AI ·
Blog/Moonshot AI
· 2026-07-16
KimiCode harness. DeepSWE v1.1 tasks.
67.5%
7
Kimi K3
Moonshot AI ·
Blog/Moonshot AI
· 2026-07-16
Self-reported with KimiCode harness on DeepSWE v1.1. First open 3T-class model; trails GPT-5.6 Sol (72.7) and Fable 5 (69.7).
67.3%
8
GPT-5.6 Luna
OpenAI ·
Blog/OpenAI
· 2026-07-09
Luna matches GPT-5.5 (67%) on DeepSWE at ~1/4 the cost.
67.2%
9
GPT-5.5
OpenAI ·
Blog/OpenAI
· 2026-06-26
Score as cited in Kimi K3 blog (source: OpenAI official).
67.0%
10
Grok 4.5
xAI ·
Blog/xAI
· 2026-07-08
DeepSWE 1.0 (v1.0), each model in its provider's harness. xAI's own run.
62.0%
11
Claude Opus 4.8
Anthropic ·
Blog/Anthropic
· 2026-06-17
Score as cited in Kimi K3 blog (source: Anthropic official).
59.0%
12
Claude Sonnet 5
Anthropic ·
Datacurve/DeepSWE leaderboard
· 2026-07-17
Anthropic Claude Sonnet 5 [max]. mini-swe-agent harness. New model on DeepSWE leaderboard.
54.0%
13
Grok 4.5
xAI ·
Blog/xAI
· 2026-07-08
DeepSWE 1.1, mini-swe-agent harness run by DataCurve. Independent eval.
53.0%
14
Muse Spark 1.1
Meta ·
Datacurve/DeepSWE leaderboard
· 2026-07-17
Meta Muse Spark 1.1 [xhigh]. mini-swe-agent harness. New Meta model on DeepSWE leaderboard.
53.0%
15
GPT-5.4
OpenAI ·
Datacurve/DeepSWE leaderboard
· 2026-07-17
OpenAI GPT-5.4 [xhigh]. mini-swe-agent harness. Older frontier model on DeepSWE leaderboard.
52.0%
16
GLM-5.2
Z.ai ·
Blog/Z.ai
· 2026-06-13
Score as cited in Kimi K3 blog (source: Z.ai official).
46.2%
17
GLM-5.2
Z.ai ·
Datacurve/DeepSWE leaderboard
· 2026-07-17
GLM-5.2 [max]. mini-swe-agent harness, v1.1. Lower than provider-reported v1.0 score.
44.0%
18
Gemini 3.5 Flash
Google DeepMind ·
Datacurve/DeepSWE leaderboard
· 2026-07-17
Gemini 3.5 Flash [medium]. mini-swe-agent harness. Long-horizon coding is a weakness.
37.0%
19
Kimi K2.7 Code
Moonshot AI ·
Datacurve/DeepSWE leaderboard
· 2026-07-17
Kimi K2.7 Code. mini-swe-agent harness. Older coding-specialist model.
31.0%
20
Claude Sonnet 4.6
Anthropic ·
Datacurve/DeepSWE leaderboard
· 2026-07-17
Claude Sonnet 4.6 [high]. mini-swe-agent harness. Previous-gen Sonnet.
30.0%
21
Hunyuan Hy3
Tencent ·
HuggingFace/tencent-Hy3-modelcard
· 2026-07-06
DeepSWE. Newer agent benchmark; Hy3 trails Kimi K3 (67.3).
28.0%
22
Gemini 3.1 Pro
Google DeepMind ·
Datacurve/DeepSWE leaderboard
· 2026-07-17
Gemini 3.1 Pro [high]. mini-swe-agent harness. Bottom of frontier leaderboard.
12.0%