Where do these coding results come from?

Both coding agents worked the same 24 merged changes from the private Stet-AI/stet repository, each in the real repo as it was before the change. The repo’s tests and judges graded every patch against the merged change; no public benchmark scores are used.

Are reasoning settings combined into one model score?

No. Each effort pairing keeps its own recorded quality call, head-to-head record, cost and agent time. Incompatible scores are never averaged.

GPT 6 Luna vs SWE-2 Max

GPT 6 Luna faster in both pairings. SWE-2 Max leads on quality in every pairing. 2 pairings; 24 merged changes in Stet-AI/stet. Compare the individual calls below to choose which setup to investigate.

What this comparison is

1 real repositoryStet-AI/stet, private; not public benchmark scores
24 merged changeseach run end to end by the coding agent from the repo as it was before the change
Graded against the merged changeby the repo’s own tests and by judges comparing each patch with it
Quality, cost and timea smaller sample than public benchmarks, but real agent work on a real codebase

Compare coding models Low vs high vs xhigh/max reasoning for coding agents

Recorded setup pairings

Each row is a separate recorded comparison. Scores are not pooled or averaged across effort settings. W/D/L means wins, draws and losses; undecided judgments remain undecided.

Stet score: A setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field.

Setup pairing	Recorded quality call and record	Quality evidence	Cost per task	Agent time per task
GPT 6 Luna max vs SWE-2 Max highGPT 6 Luna max · max reasoningSWE-2 Max high · high reasoning24/24 tasks observed by both setups.	SWE-2 Max high leads on quality: won 7, tied 15, lost 1, 1 undecided head to head; GPT 6 Luna max leads on agent time: 1.1× faster; SWE-2 Max high’s cost wasn’t recorded.Head to head, SWE-2 Max high won 7, tied 15 and lost 1 against GPT 6 Luna max, with 1 undecided.Not yet conclusive across all 136 comparisons on the board.	Clear lead 7–1 head to head, 15 tiedGPT 6 Luna max: 46.0 (95% range 40.4–51.6)SWE-2 Max high: 58.0 (95% range 49.7–66.8)	GPT 6 Luna max: $0.37 SWE-2 Max high: not recorded	GPT 6 Luna max: 59.7 min SWE-2 Max high: 68.3 min
GPT 6 Luna low vs SWE-2 Max highGPT 6 Luna low · low reasoningSWE-2 Max high · high reasoning24/24 tasks observed by both setups.	SWE-2 Max high leads on quality: won 10, tied 12, lost 1, 1 undecided head to head; GPT 6 Luna low leads on agent time: 9.9× faster; SWE-2 Max high’s cost wasn’t recorded.Head to head, SWE-2 Max high won 10, tied 12 and lost 1 against GPT 6 Luna low, with 1 undecided.Not yet conclusive across all 136 comparisons on the board.	Clear lead 10–1 head to head, 12 tiedGPT 6 Luna low: 37.8 (95% range 27.8–47.7)SWE-2 Max high: 58.0 (95% range 49.7–66.8)	GPT 6 Luna low: $0.05 SWE-2 Max high: not recorded	GPT 6 Luna low: 6.9 min SWE-2 Max high: 68.3 min

Setup pairing

Recorded quality call and record

Quality evidence

Cost per task

Agent time per task

GPT 6 Luna max vs SWE-2 Max highGPT 6 Luna max · max reasoningSWE-2 Max high · high reasoning24/24 tasks observed by both setups.

SWE-2 Max high leads on quality: won 7, tied 15, lost 1, 1 undecided head to head; GPT 6 Luna max leads on agent time: 1.1× faster; SWE-2 Max high’s cost wasn’t recorded.Head to head, SWE-2 Max high won 7, tied 15 and lost 1 against GPT 6 Luna max, with 1 undecided.Not yet conclusive across all 136 comparisons on the board.

Clear lead
7–1 head to head, 15 tiedGPT 6 Luna max: 46.0 (95% range 40.4–51.6)SWE-2 Max high: 58.0 (95% range 49.7–66.8)

GPT 6 Luna max: $0.37

SWE-2 Max high: not recorded

GPT 6 Luna max: 59.7 min

SWE-2 Max high: 68.3 min

GPT 6 Luna low vs SWE-2 Max highGPT 6 Luna low · low reasoningSWE-2 Max high · high reasoning24/24 tasks observed by both setups.

SWE-2 Max high leads on quality: won 10, tied 12, lost 1, 1 undecided head to head; GPT 6 Luna low leads on agent time: 9.9× faster; SWE-2 Max high’s cost wasn’t recorded.Head to head, SWE-2 Max high won 10, tied 12 and lost 1 against GPT 6 Luna low, with 1 undecided.Not yet conclusive across all 136 comparisons on the board.

Clear lead
10–1 head to head, 12 tiedGPT 6 Luna low: 37.8 (95% range 27.8–47.7)SWE-2 Max high: 58.0 (95% range 49.7–66.8)

GPT 6 Luna low: $0.05

SWE-2 Max high: not recorded

GPT 6 Luna low: 6.9 min

SWE-2 Max high: 68.3 min

Questions about these results

Which is better for coding, GPT 6 Luna or SWE-2 Max?: GPT 6 Luna faster in both pairings. SWE-2 Max leads on quality in every pairing. 2 pairings; 24 merged changes in Stet-AI/stet. Compare the individual calls below to choose which setup to investigate.
Where do these coding results come from?: Both coding agents worked the same 24 merged changes from the private Stet-AI/stet repository, each in the real repo as it was before the change. The repo’s tests and judges graded every patch against the merged change; no public benchmark scores are used.
Are reasoning settings combined into one model score?: No. Each effort pairing keeps its own recorded quality call, head-to-head record, cost and agent time. Incompatible scores are never averaged.