Where do these coding results come from?

Both coding agents worked the same 24 merged changes from the private Stet-AI/stet repository, each in the real repo as it was before the change. The repo’s tests and judges graded every patch against the merged change; no public benchmark scores are used.

Are reasoning settings combined into one model score?

No. Each effort pairing keeps its own recorded quality call, head-to-head record, cost and agent time. Incompatible scores are never averaged.

GPT 5.6 Luna vs GPT 6 Luna

GPT 5.6 Luna leads on quality in 2 of 4 pairings; the rest show no clear difference. 4 pairings; 24 merged changes in Stet-AI/stet. Compare the individual calls below to choose which setup to investigate.

What this comparison is

1 real repositoryStet-AI/stet, private; not public benchmark scores
24 merged changeseach run end to end by the coding agent from the repo as it was before the change
Graded against the merged changeby the repo’s own tests and by judges comparing each patch with it
Quality, cost and timea smaller sample than public benchmarks, but real agent work on a real codebase

Compare coding models Low vs high vs xhigh/max reasoning for coding agents

Recorded setup pairings

Each row is a separate recorded comparison. Scores are not pooled or averaged across effort settings. W/D/L means wins, draws and losses; undecided judgments remain undecided.

Stet score: A setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field.

Setup pairing	Recorded quality call and record	Quality evidence	Cost per task	Agent time per task
GPT 5.6 Luna max vs GPT 6 Luna maxGPT 5.6 Luna max · max reasoningGPT 6 Luna max · max reasoning24/24 tasks observed by both setups.	GPT 5.6 Luna max has a slight quality edge, 2–0 head to head with 21 tied and 1 undecided; GPT 6 Luna max leads on cost: 2.4× cheaper; GPT 5.6 Luna max leads on agent time: 2.1× faster.Head to head, GPT 5.6 Luna max won 2, tied 21 and lost 0 against GPT 6 Luna max, with 1 undecided.	Slight edge 2–0 head to head, 21 tiedGPT 5.6 Luna max: 49.7 (95% range 44.0–55.6)GPT 6 Luna max: 46.0 (95% range 40.4–51.6)	GPT 5.6 Luna max: $0.90 GPT 6 Luna max: $0.37	GPT 5.6 Luna max: 28.5 min GPT 6 Luna max: 59.7 min
GPT 5.6 Luna max vs GPT 6 Luna lowGPT 5.6 Luna max · max reasoningGPT 6 Luna low · low reasoning24/24 tasks observed by both setups.	GPT 5.6 Luna max leads on quality, 6–1 head to head with 16 tied and 1 undecided (directional at this sample size); GPT 6 Luna low leads on cost and agent time: 16.8× cheaper and 4.1× faster.Head to head, GPT 5.6 Luna max won 6, tied 16 and lost 1 against GPT 6 Luna low, with 1 undecided.	Directional lead at this sample size 6–1 head to head, 16 tiedGPT 5.6 Luna max: 49.7 (95% range 44.0–55.6)GPT 6 Luna low: 37.8 (95% range 27.8–47.7)	GPT 5.6 Luna max: $0.90 GPT 6 Luna low: $0.05	GPT 5.6 Luna max: 28.5 min GPT 6 Luna low: 6.9 min
GPT 5.6 Luna low vs GPT 6 Luna maxGPT 5.6 Luna low · codex · low reasoningGPT 6 Luna max · max reasoning24/24 tasks observed by both setups.	No clear quality difference: GPT 6 Luna max won 1, GPT 5.6 Luna low won 0, 22 tied, 1 undecided; GPT 5.6 Luna low leads on cost and agent time: 7.3× cheaper and 8.9× faster.Head to head, GPT 6 Luna max won 1, tied 22 and lost 0 against GPT 5.6 Luna low, with 1 undecided.	No clear difference 0–1 head to head, 22 tiedGPT 5.6 Luna low: 44.6 (95% range 38.3–50.3)GPT 6 Luna max: 46.0 (95% range 40.4–51.6)	GPT 5.6 Luna low: $0.05 GPT 6 Luna max: $0.37	GPT 5.6 Luna low: 6.7 min GPT 6 Luna max: 59.7 min
GPT 5.6 Luna low vs GPT 6 Luna lowGPT 5.6 Luna low · codex · low reasoningGPT 6 Luna low · low reasoning24/24 tasks observed by both setups.	GPT 5.6 Luna low leads on quality, 6–1 head to head with 16 tied and 1 undecided (directional at this sample size); GPT 5.6 Luna low leads on agent time: a little faster; they tie on cost.Head to head, GPT 5.6 Luna low won 6, tied 16 and lost 1 against GPT 6 Luna low, with 1 undecided.	Directional lead at this sample size 6–1 head to head, 16 tiedGPT 5.6 Luna low: 44.6 (95% range 38.3–50.3)GPT 6 Luna low: 37.8 (95% range 27.8–47.7)	GPT 5.6 Luna low: $0.05 GPT 6 Luna low: $0.05	GPT 5.6 Luna low: 6.7 min GPT 6 Luna low: 6.9 min

Questions about these results

Which is better for coding, GPT 5.6 Luna or GPT 6 Luna?: GPT 5.6 Luna leads on quality in 2 of 4 pairings; the rest show no clear difference. 4 pairings; 24 merged changes in Stet-AI/stet. Compare the individual calls below to choose which setup to investigate.
Where do these coding results come from?: Both coding agents worked the same 24 merged changes from the private Stet-AI/stet repository, each in the real repo as it was before the change. The repo’s tests and judges graded every patch against the merged change; no public benchmark scores are used.
Are reasoning settings combined into one model score?: No. Each effort pairing keeps its own recorded quality call, head-to-head record, cost and agent time. Incompatible scores are never averaged.