Skip to the comparison

GPT 6 Luna vs SWE-2

GPT 6 Luna cheaper and faster in both pairings. No clear quality difference in any pairing.

2 pairings; 24 merged changes in Stet-AI/stet. Data from

Recorded setup pairings

GPT 6 Luna max vs SWE-2 max

GPT 6 Luna max leads on speed and cost; no clear quality difference.

Quality
Even
No clear difference
1–9 head to head, 14 tied
Speed
1.1× faster
GPT 6 Luna max
59.7 min vs 68.3 min per task
Cost
7.5× cheaper
GPT 6 Luna max
$0.37 vs $2.76 per task; $2.76 from 23/24 tasks

Stet score

GPT 6 Luna maxcodex · max reasoningRank 13 of 18

47.795% range 42.0–53.5

SWE-2 maxdevin · max reasoningRank 1 of 18

62.395% range 53.4–71.6

Their 95% score ranges overlap.

Head-to-head matchups

GPT 6 Luna max won1

Tied14

SWE-2 max won9

SWE-2 max’s win share 67%, 95% range 56–77%Ties are within judge noise.

GPT 6 Luna low vs SWE-2 max

GPT 6 Luna low leads on speed and cost; no clear quality difference.

20/24 tasks run by both setups

Quality
Even
No clear difference
0–8 head to head, 12 tied, 4 undecided
Speed
9.9× faster
GPT 6 Luna low
6.9 min vs 68.3 min per task
Cost
52× cheaper
GPT 6 Luna low
$0.05 vs $2.76 per task; $2.76 from 23/24 tasks

Stet score

GPT 6 Luna lowcodex · low reasoningRank 18 of 18

39.895% range 31.3–47.9

SWE-2 maxdevin · max reasoningRank 1 of 18

62.395% range 53.4–71.6

Their 95% score ranges don’t overlap.

Head-to-head matchups

GPT 6 Luna low won0

Tied12

4 undecided

SWE-2 max won8

SWE-2 max’s win share 70%, 95% range 60–81%Ties are within judge noise.

Questions about these results

Which is better for coding, GPT 6 Luna or SWE-2?
GPT 6 Luna cheaper and faster in both pairings. No clear quality difference in any pairing. 2 pairings; 24 merged changes in Stet-AI/stet.
Where do these coding results come from?
Both coding agents worked the same 24 merged changes from the private Stet-AI/stet repository, each in the real repo as it was before the change. The repo’s tests and judges graded every patch against the merged change; no public benchmark scores are used.
Are reasoning settings combined into one model score?
No. Each effort pairing keeps its own recorded quality call, head-to-head record, cost and agent time. Incompatible scores are never averaged.