Skip to the comparison

GPT 5.6 Luna vs SWE-2

GPT 5.6 Luna cheaper and faster in both pairings. No clear quality difference in any pairing.

2 pairings; 24 merged changes in Stet-AI/stet. Data from

Recorded setup pairings

GPT 5.6 Luna max vs SWE-2 max

GPT 5.6 Luna max leads on speed and cost; no clear quality difference.

Quality
Even
No clear difference
1–9 head to head, 14 tied
Speed
2.4× faster
GPT 5.6 Luna max
28.5 min vs 68.3 min per task
Cost
3.1× cheaper
GPT 5.6 Luna max
$0.90 vs $2.76 per task; $2.76 from 23/24 tasks

Stet score

GPT 5.6 Luna maxcodex · max reasoningRank 15 of 18

45.495% range 40.0–50.2

SWE-2 maxdevin · max reasoningRank 1 of 18

62.395% range 53.4–71.6

Their 95% score ranges don’t overlap.

Head-to-head matchups

GPT 5.6 Luna max won1

Tied14

SWE-2 max won9

SWE-2 max’s win share 67%, 95% range 56–77%Ties are within judge noise.

GPT 5.6 Luna low vs SWE-2 max

GPT 5.6 Luna low leads on speed and cost; no clear quality difference.

23/24 tasks run by both setups

Quality
Even
No clear difference
1–9 head to head, 13 tied, 1 undecided
Speed
10× faster
GPT 5.6 Luna low
6.7 min vs 68.3 min per task
Cost
54× cheaper
GPT 5.6 Luna low
$0.05 vs $2.76 per task; $2.76 from 23/24 tasks

Stet score

GPT 5.6 Luna lowcodex · low reasoningRank 16 of 18

44.795% range 37.7–51.0

SWE-2 maxdevin · max reasoningRank 1 of 18

62.395% range 53.4–71.6

Their 95% score ranges don’t overlap.

Head-to-head matchups

GPT 5.6 Luna low won1

Tied13

1 undecided

SWE-2 max won9

SWE-2 max’s win share 67%, 95% range 56–79%Ties are within judge noise.

Questions about these results

Which is better for coding, GPT 5.6 Luna or SWE-2?
GPT 5.6 Luna cheaper and faster in both pairings. No clear quality difference in any pairing. 2 pairings; 24 merged changes in Stet-AI/stet.
Where do these coding results come from?
Both coding agents worked the same 24 merged changes from the private Stet-AI/stet repository, each in the real repo as it was before the change. The repo’s tests and judges graded every patch against the merged change; no public benchmark scores are used.
Are reasoning settings combined into one model score?
No. Each effort pairing keeps its own recorded quality call, head-to-head record, cost and agent time. Incompatible scores are never averaged.