GPT 5.6 Luna max vs Opus 4.8 medium
Opus 4.8 medium leads on speed; GPT 5.6 Luna max on cost; quality is too close to call.
- Quality
- +0.9 pts
- GPT 5.6 Luna max
- 5–4 head to head, 14 tied
- Speed
- 1.2× faster
- Opus 4.8 medium
- 23.3 min vs 28.5 min per task
- Cost
- 7.0× cheaper
- GPT 5.6 Luna max
- $0.90 vs $6.27 per task
Stet score
GPT 5.6 Luna maxmax reasoningRank 10 of 17
49.795% range 44.0–55.60.9 points higher
Opus 4.8 mediummedium reasoningRank 11 of 17
48.895% range 39.7–58.0
Quality
Head-to-head matchups
GPT 5.6 Luna max won5
Tied14
Opus 4.8 medium won4
Task by task
GPT 5.6 Luna max solved 0 tasks that Opus 4.8 medium didn’t, and Opus 4.8 medium solved 3 that GPT 5.6 Luna max didn’t. Both solved 10; neither solved 11.
stet-commit-f8ae1177Solved by 10 of 17 setups
- GPT 5.6 Luna maxFailed
- Opus 4.8 mediumPassed tests
A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change, as in the leaderboard’s Solved column. Stet score also weighs code quality, so it can order the two differently.
All 24 compared tasks
| Task | GPT 5.6 Luna max | Opus 4.8 medium | Solved by |
|---|---|---|---|
stet-3169aefa | Passed tests | Passed tests | Both |
stet-89dfbc27 | Passed tests | Passed tests | Both |
stet-b28f4fe7 | Passed tests | Passed tests | Both |
stet-232b50ba | Passed tests | Passed tests | Both |
stet-5276aa3f | Passed tests | Passed tests | Both |
stet-commit-bbbbae09 | Passed tests | Passed tests | Both |
stet-66762b4c | Equivalent to the merged fix | Passed tests | Both |
stet-commit-5d766dde | Equivalent to the merged fix | Equivalent to the merged fix | Both |
stet-43493c52 | Passed tests | Passed tests | Both |
stet-commit-f0caded9 | Equivalent to the merged fix | Equivalent to the merged fix | Both |
stet-commit-f8ae1177 | Failed | Passed tests | Opus 4.8 medium only |
stet-15439c21 | Failed | Failed | Neither |
stet-2450ca2d | Failed | Failed | Neither |
stet-commit-342d0ac0 | Failed | Failed | Neither |
stet-6f84e978 | Failed | Passed tests | Opus 4.8 medium only |
stet-commit-60a0e6c2 | Failed | Passed tests | Opus 4.8 medium only |
stet-commit-5e075641 | Failed | Failed | Neither |
stet-commit-2826765e | Failed | Failed | Neither |
stet-commit-a8716b88 | Failed | Failed | Neither |
stet-commit-e928166f | Failed | Failed | Neither |
stet-commit-30506f8e | Failed | Failed | Neither |
stet-commit-87814583 | Failed | Failed | Neither |
stet-commit-d2ab0c25 | Failed | Failed | Neither |
stet-commit-e41c2e2d | Failed | Failed | Neither |
Side by side
| Metric | GPT 5.6 Luna max | Opus 4.8 medium |
|---|---|---|
| Reasoning effort | max | medium |
| Rank | 10 of 17 | 11 of 17 |
| Stet score | 49.795% range 44.0–55.6 | 48.895% range 39.7–58.0 |
| Solved | 10/247 passed, 3 judged equivalent | 13/2411 passed, 2 judged equivalent |
| Cost / task | $0.90 | $6.27 |
| Agent time / task | 28.5 min | 23.3 min |
See which setup wins on your repo
Stet replays your merged PRs, scores quality above pass/fail, and delivers comparison reports through your agent.
Get startedAbout this data
- 24 tasks, each a merged change replayed from the repo as it was just before the merge.
- Data from .
- Both setups come from the same leaderboard run of 17 setups:
bmk_d4ba310fdf50db4ccb2092e3.