GPT 6 Luna low vs Pareto 26.9 medium
Pareto 26.9 medium leads on quality; GPT 6 Luna low on speed and cost.
- Quality
- +17.4 pts
- Pareto 26.9 medium
- 8–1 head to head, 14 tied
- Not yet conclusive across all 136 comparisons on the board.
- Speed
- 5.7× faster
- GPT 6 Luna low
- 6.9 min vs 39.2 min per task
- Cost
- 112.1× cheaper
- GPT 6 Luna low
- $0.05 vs $5.98 per task
Stet score
GPT 6 Luna lowlow reasoningRank 17 of 17
37.895% range 27.8–47.7
Pareto 26.9 mediummedium reasoningRank 3 of 17
55.295% range 50.5–60.517.4 points higher
Quality
Head-to-head matchups
GPT 6 Luna low won1
Tied14
Pareto 26.9 medium won8
Task by task
GPT 6 Luna low solved 1 task that Pareto 26.9 medium didn’t, and Pareto 26.9 medium solved 7 that GPT 6 Luna low didn’t. Both solved 4; neither solved 12.
stet-5276aa3fSolved by 16 of 17 setups
- GPT 6 Luna lowFailed
- Pareto 26.9 mediumPassed tests
A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change, as in the leaderboard’s Solved column. Stet score also weighs code quality, so it can order the two differently.
All 24 compared tasks
| Task | GPT 6 Luna low | Pareto 26.9 medium | Solved by |
|---|---|---|---|
stet-3169aefa | Passed tests | Passed tests | Both |
stet-89dfbc27 | Passed tests | Passed tests | Both |
stet-b28f4fe7 | Passed tests | Passed tests | Both |
stet-232b50ba | Passed tests | Passed tests | Both |
stet-5276aa3f | Failed | Passed tests | Pareto 26.9 medium only |
stet-commit-bbbbae09 | No change made | Passed tests | Pareto 26.9 medium only |
stet-66762b4c | Failed | Passed tests | Pareto 26.9 medium only |
stet-commit-5d766dde | No change made | Failed | Neither |
stet-43493c52 | Failed | Passed tests | Pareto 26.9 medium only |
stet-commit-f0caded9 | Failed | Equivalent to the merged fix | Pareto 26.9 medium only |
stet-commit-f8ae1177 | No change made | Passed tests | Pareto 26.9 medium only |
stet-15439c21 | Passed tests | Failed | GPT 6 Luna low only |
stet-2450ca2d | Failed | Passed tests | Pareto 26.9 medium only |
stet-commit-342d0ac0 | Failed | Failed | Neither |
stet-6f84e978 | Failed | Failed | Neither |
stet-commit-60a0e6c2 | No change made | Failed | Neither |
stet-commit-5e075641 | Failed | Failed | Neither |
stet-commit-2826765e | Failed | Failed | Neither |
stet-commit-a8716b88 | Failed | Failed | Neither |
stet-commit-e928166f | Failed | Failed | Neither |
stet-commit-30506f8e | Failed | Failed | Neither |
stet-commit-87814583 | Failed | Failed | Neither |
stet-commit-d2ab0c25 | Failed | Failed | Neither |
stet-commit-e41c2e2d | Failed | Failed | Neither |
Side by side
| Metric | GPT 6 Luna low | Pareto 26.9 medium |
|---|---|---|
| Reasoning effort | low | medium |
| Rank | 17 of 17 | 3 of 17 |
| Stet score | 37.895% range 27.8–47.7 | 55.295% range 50.5–60.5 |
| Solved | 5/245 passed, 0 judged equivalent | 11/2410 passed, 1 judged equivalent |
| Cost / task | $0.05 | $5.98 |
| Agent time / task | 6.9 min | 39.2 min |
See which setup wins on your repo
Stet replays your merged PRs, scores quality above pass/fail, and delivers comparison reports through your agent.
Get startedAbout this data
- 24 tasks, each a merged change replayed from the repo as it was just before the merge.
- Data from .
- Both setups come from the same leaderboard run of 17 setups:
bmk_d4ba310fdf50db4ccb2092e3.