GPT 6 Luna vs Pareto 26.9
GPT 6 Luna cheaper in both pairings. Pareto 26.9 leads on quality in every pairing. 2 pairings; 24 merged changes in Stet-AI/stet. Compare the individual calls below to choose which setup to investigate.
What this comparison is
- 1 real repositoryStet-AI/stet, private; not public benchmark scores
- 24 merged changeseach run end to end by the coding agent from the repo as it was before the change
- Graded against the merged changeby the repo’s own tests and by judges comparing each patch with it
- Quality, cost and timea smaller sample than public benchmarks, but real agent work on a real codebase
Compare coding modelsLow vs high vs xhigh/max reasoning for coding agents
Recorded setup pairings
Each row is a separate recorded comparison. Scores are not pooled or averaged across effort settings. W/D/L means wins, draws and losses; undecided judgments remain undecided.
Stet score: A setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field.
| Setup pairing | Recorded quality call and record | Quality evidence | Cost per task | Agent time per task |
|---|---|---|---|---|
| GPT 6 Luna max vs Pareto 26.9 mediumGPT 6 Luna max · max reasoningPareto 26.9 medium · medium reasoning24/24 tasks observed by both setups. | Pareto 26.9 medium leads on quality and agent time: won 5, tied 18, lost 0, 1 undecided head to head and 1.5× faster; GPT 6 Luna max leads on cost: 16.2× cheaper.Head to head, Pareto 26.9 medium won 5, tied 18 and lost 0 against GPT 6 Luna max, with 1 undecided.Not yet conclusive across all 136 comparisons on the board. | Clear lead 5–0 head to head, 18 tiedGPT 6 Luna max: 46.0 (95% range 40.4–51.6)Pareto 26.9 medium: 55.2 (95% range 50.5–60.5) | GPT 6 Luna max: $0.37 Pareto 26.9 medium: $5.98 | GPT 6 Luna max: 59.7 min Pareto 26.9 medium: 39.2 min |
| GPT 6 Luna low vs Pareto 26.9 mediumGPT 6 Luna low · low reasoningPareto 26.9 medium · medium reasoning24/24 tasks observed by both setups. | Pareto 26.9 medium leads on quality: won 8, tied 14, lost 1, 1 undecided head to head; GPT 6 Luna low leads on cost and agent time: 112.1× cheaper and 5.7× faster.Head to head, Pareto 26.9 medium won 8, tied 14 and lost 1 against GPT 6 Luna low, with 1 undecided.Not yet conclusive across all 136 comparisons on the board. | Clear lead 8–1 head to head, 14 tiedGPT 6 Luna low: 37.8 (95% range 27.8–47.7)Pareto 26.9 medium: 55.2 (95% range 50.5–60.5) | GPT 6 Luna low: $0.05 Pareto 26.9 medium: $5.98 | GPT 6 Luna low: 6.9 min Pareto 26.9 medium: 39.2 min |
Questions about these results
- Which is better for coding, GPT 6 Luna or Pareto 26.9?
- GPT 6 Luna cheaper in both pairings. Pareto 26.9 leads on quality in every pairing. 2 pairings; 24 merged changes in Stet-AI/stet. Compare the individual calls below to choose which setup to investigate.
- Where do these coding results come from?
- Both coding agents worked the same 24 merged changes from the private Stet-AI/stet repository, each in the real repo as it was before the change. The repo’s tests and judges graded every patch against the merged change; no public benchmark scores are used.
- Are reasoning settings combined into one model score?
- No. Each effort pairing keeps its own recorded quality call, head-to-head record, cost and agent time. Incompatible scores are never averaged.