GPT 5.6 Luna low vs Muse Spark 1.3 high
GPT 5.6 Luna low leads on speed and cost; quality is too close to call.
- Quality
- +2.8 pts
- Muse Spark 1.3 high
- 1–0 head to head, 10 tied
- Speed
- 2.0× faster
- GPT 5.6 Luna low
- 6.7 min vs 13.2 min per task
- Cost
- 2.0× cheaper
- GPT 5.6 Luna low
- $0.05 vs $0.10 per task
Stet score
GPT 5.6 Luna lowcodex · low reasoningRank 15 of 17
45.095% range 39.1–50.4
Muse Spark 1.3 highhigh reasoningRank —
47.895% range 43.4–51.52.8 points higher
Quality
Head-to-head matchups
GPT 5.6 Luna low won0
Tied10
Muse Spark 1.3 high won1
Task by task
GPT 5.6 Luna low solved 2 tasks that Muse Spark 1.3 high didn’t, and Muse Spark 1.3 high solved 2 that GPT 5.6 Luna low didn’t. Both solved 2; neither solved 5.
stet-commit-bbbbae09Solved by 15 of 17 setups
- GPT 5.6 Luna lowFailed
- Muse Spark 1.3 highPassed tests
A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change, as in the leaderboard’s Solved column. Stet score also weighs code quality, so it can order the two differently. 13 of the 24 tasks are left out because one of the two has no scored result on them.
All 11 compared tasks
| Task | GPT 5.6 Luna low | Muse Spark 1.3 high | Solved by |
|---|---|---|---|
stet-232b50ba | Passed tests | Passed tests | Both |
stet-b28f4fe7 | Passed tests | Passed tests | Both |
stet-commit-bbbbae09 | Failed | Passed tests | Muse Spark 1.3 high only |
stet-commit-f0caded9 | Equivalent to the merged fix | Failed | GPT 5.6 Luna low only |
stet-15439c21 | Equivalent to the merged fix | Failed | GPT 5.6 Luna low only |
stet-2450ca2d | Failed | Equivalent to the merged fix | Muse Spark 1.3 high only |
stet-commit-60a0e6c2 | Failed | Failed | Neither |
stet-commit-5e075641 | Failed | Failed | Neither |
stet-commit-2826765e | Failed | Failed | Neither |
stet-commit-e928166f | Failed | Failed | Neither |
stet-commit-d2ab0c25 | Failed | Failed | Neither |
Side by side
| Metric | GPT 5.6 Luna low | Muse Spark 1.3 high |
|---|---|---|
| Harness | codex | — |
| Reasoning effort | low | high |
| Rank | 15 of 17 | —Position withheld by the report |
| Stet score | 45.095% range 39.1–50.4 | 47.895% range 43.4–51.5 |
| Solved | 10/246 passed, 4 judged equivalent | 4/243 passed, 1 judged equivalent |
| Cost / task | $0.05 | $0.10 |
| Agent time / task | 6.7 min | 13.2 min |
See which setup wins on your repo
Stet replays your merged PRs, scores quality above pass/fail, and delivers comparison reports through your agent.
Run it on your repoAbout this data
- 24 tasks, each a merged change replayed from the repo as it was just before the merge.
- Data from .
- Both setups come from the same leaderboard run of 17 setups:
bmk_d4ba310fdf50db4ccb2092e3.