Muse Spark 1.3 high vs Opus 4.8 medium
No clear quality winner: Opus 4.8 medium won 11, Muse Spark 1.3 high won 7, 6 tied; Muse Spark 1.3 high leads on cost and agent time: 70.4× cheaper and 1.8× faster.
Stet score
Muse Spark 1.3 highmuse · high reasoningRank 14 of 16
31.995% range 23.5–40.6
Opus 4.8 mediumclaude-code · medium reasoningRank 12 of 16
39.295% range 29.0–49.77.3 points higher
Quality
Head-to-head matchups
Muse Spark 1.3 high won7
Tied6
Opus 4.8 medium won11
Task by task
Muse Spark 1.3 high solved 1 task that Opus 4.8 medium didn’t, and Opus 4.8 medium solved 4 that Muse Spark 1.3 high didn’t. Both solved 9; neither solved 10.
Muse Spark 1.3 high only 1
stet-2450ca2dEquivalent to the merged fix
Opus 4.8 medium only 4
stet-15439c21Passed testsstet-commit-f0caded9Equivalent to the merged fixstet-6f84e978Passed testsstet-commit-60a0e6c2Passed tests
A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change, as in the leaderboard’s Solved column. Stet score also weighs code quality, so it can order the two differently.
All 24 compared tasks
| Task | Muse Spark 1.3 high | Opus 4.8 medium | Solved by |
|---|---|---|---|
stet-3169aefa | Passed tests | Passed tests | Both |
stet-89dfbc27 | Passed tests | Passed tests | Both |
stet-b28f4fe7 | Passed tests | Passed tests | Both |
stet-232b50ba | Passed tests | Passed tests | Both |
stet-5276aa3f | Passed tests | Passed tests | Both |
stet-commit-bbbbae09 | Passed tests | Passed tests | Both |
stet-66762b4c | Passed tests | Passed tests | Both |
stet-15439c21 | Failed | Passed tests | Opus 4.8 medium only |
stet-commit-5d766dde | Equivalent to the merged fix | Equivalent to the merged fix | Both |
stet-43493c52 | Passed tests | Passed tests | Both |
stet-commit-f0caded9 | Failed | Equivalent to the merged fix | Opus 4.8 medium only |
stet-6f84e978 | Failed | Passed tests | Opus 4.8 medium only |
stet-2450ca2d | Equivalent to the merged fix | Failed | Muse Spark 1.3 high only |
stet-commit-342d0ac0 | Failed | Failed | Neither |
stet-commit-f8ae1177 | Failed | Failed | Neither |
stet-commit-60a0e6c2 | Failed | Passed tests | Opus 4.8 medium only |
stet-commit-5e075641 | Failed | Failed | Neither |
stet-commit-2826765e | Failed | Failed | Neither |
stet-commit-a8716b88 | Failed | Failed | Neither |
stet-commit-e928166f | Failed | Failed | Neither |
stet-commit-30506f8e | Failed | Failed | Neither |
stet-commit-87814583 | Failed | Failed | Neither |
stet-commit-d2ab0c25 | Failed | Failed | Neither |
stet-commit-e41c2e2d | Failed | Failed | Neither |
Side by side
| Metric | Muse Spark 1.3 high | Opus 4.8 medium |
|---|---|---|
| Harness | muse | claude-code |
| Reasoning effort | high | medium |
| Rank | 14 of 16 | 12 of 16 |
| Stet score | 31.995% range 23.5–40.6 | 39.295% range 29.0–49.7 |
| Solved | 10/248 passed, 2 judged equivalent | 13/2411 passed, 2 judged equivalent |
| Cost / task | $0.10 | $7.05 |
| Agent time / task | 13.2 min | 23.3 min |
See which setup wins on your repo
Stet replays your merged PRs, scores quality above pass/fail, and delivers comparison reports through your agent.
Run it on your repoAbout this data
- 24 tasks, each a merged change replayed from the repo as it was just before the merge.
- Data from .
- Both setups come from the same leaderboard run of 16 setups:
bmk_d4ba310fdf50db4ccb2092e3.