Haiku 5.5 low vs Muse Spark 1.3 high
No clear quality winner: Haiku 5.5 low won 12, Muse Spark 1.3 high won 5, 7 tied; Muse Spark 1.3 high leads on cost: 2.5× cheaper; Haiku 5.5 low leads on agent time: 1.3× faster.
Stet score
Haiku 5.5 lowclaude-code · low reasoningRank 13 of 16
38.995% range 32.6–45.47.0 points higher
Muse Spark 1.3 highmuse · high reasoningRank 14 of 16
31.995% range 23.5–40.6
Quality
Head-to-head matchups
Haiku 5.5 low won12
Tied7
Muse Spark 1.3 high won5
Task by task
Haiku 5.5 low solved 2 tasks that Muse Spark 1.3 high didn’t, and Muse Spark 1.3 high solved 1 that Haiku 5.5 low didn’t. Both solved 9; neither solved 12.
Haiku 5.5 low only 2
stet-15439c21Equivalent to the merged fixstet-commit-f0caded9Equivalent to the merged fix
Muse Spark 1.3 high only 1
stet-2450ca2dEquivalent to the merged fix
A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change, as in the leaderboard’s Solved column. Stet score also weighs code quality, so it can order the two differently.
All 24 compared tasks
| Task | Haiku 5.5 low | Muse Spark 1.3 high | Solved by |
|---|---|---|---|
stet-3169aefa | Passed tests | Passed tests | Both |
stet-89dfbc27 | Passed tests | Passed tests | Both |
stet-b28f4fe7 | Passed tests | Passed tests | Both |
stet-232b50ba | Passed tests | Passed tests | Both |
stet-5276aa3f | Passed tests | Passed tests | Both |
stet-commit-bbbbae09 | Passed tests | Passed tests | Both |
stet-66762b4c | Equivalent to the merged fix | Passed tests | Both |
stet-15439c21 | Equivalent to the merged fix | Failed | Haiku 5.5 low only |
stet-commit-5d766dde | Equivalent to the merged fix | Equivalent to the merged fix | Both |
stet-43493c52 | Passed tests | Passed tests | Both |
stet-commit-f0caded9 | Equivalent to the merged fix | Failed | Haiku 5.5 low only |
stet-6f84e978 | Failed | Failed | Neither |
stet-2450ca2d | Failed | Equivalent to the merged fix | Muse Spark 1.3 high only |
stet-commit-342d0ac0 | Failed | Failed | Neither |
stet-commit-f8ae1177 | Failed | Failed | Neither |
stet-commit-60a0e6c2 | Failed | Failed | Neither |
stet-commit-5e075641 | Failed | Failed | Neither |
stet-commit-2826765e | Failed | Failed | Neither |
stet-commit-a8716b88 | Failed | Failed | Neither |
stet-commit-e928166f | Failed | Failed | Neither |
stet-commit-30506f8e | Failed | Failed | Neither |
stet-commit-87814583 | Failed | Failed | Neither |
stet-commit-d2ab0c25 | Failed | Failed | Neither |
stet-commit-e41c2e2d | Failed | Failed | Neither |
Side by side
| Metric | Haiku 5.5 low | Muse Spark 1.3 high |
|---|---|---|
| Harness | claude-code | muse |
| Reasoning effort | low | high |
| Rank | 13 of 16 | 14 of 16 |
| Stet score | 38.995% range 32.6–45.4 | 31.995% range 23.5–40.6 |
| Solved | 11/247 passed, 4 judged equivalent | 10/248 passed, 2 judged equivalent |
| Cost / task | $0.25 | $0.10 |
| Agent time / task | 9.9 min | 13.2 min |
See which setup wins on your repo
Stet replays your merged PRs, scores quality above pass/fail, and delivers comparison reports through your agent.
Run it on your repoAbout this data
- 24 tasks, each a merged change replayed from the repo as it was just before the merge.
- Data from .
- Both setups come from the same leaderboard run of 16 setups:
bmk_d4ba310fdf50db4ccb2092e3.