Skip to the comparison
STET
Join the waitlist

Haiku 5.5 max vs Opus 5 medium

No clear quality winner: Opus 5 medium won 9, Haiku 5.5 max won 5, 9 tied, 1 undecided; Haiku 5.5 max leads on cost: 1.8× cheaper; Opus 5 medium leads on agent time: 2.5× faster.

24 tasksData from

Stet score

Haiku 5.5 maxclaude-code · max reasoningRank 7 of 15

57.195% range 45.9–67.6

Opus 5 mediummedium reasoningRank 4 of 15

63.795% range 53.3–73.66.6 points higher

A directional read: their 95% score ranges overlap.

Cost / task

Haiku 5.5 max$3.851.8× cheaper
Opus 5 medium$6.78

Agent time / task

Haiku 5.5 max58.4 min
Opus 5 medium23.2 min2.5× faster

Quality

Head-to-head matchups

Haiku 5.5 max won5

Tied91 undecided

Opus 5 medium won9

Opus 5 medium’s win share 59%, 95% range 43–75%Ties are within judge noise.

Task by task

Haiku 5.5 max solved 2 tasks that Opus 5 medium didn’t, and Opus 5 medium solved 4 that Haiku 5.5 max didn’t. Both solved 10; neither solved 7.

Haiku 5.5 max only 2

  • stet-commit-f8ae1177Passed tests
  • stet-commit-342d0ac0Equivalent to the merged fix

Opus 5 medium only 4

  • stet-232b50baPassed tests
  • stet-66762b4cPassed tests
  • stet-43493c52Passed tests
  • stet-commit-2826765eEquivalent to the merged fix

A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change, as in the leaderboard’s Solved column. Stet score also weighs code quality, so it can order the two differently. 1 of the 24 tasks is left out because one of the two has no scored result on it.

All 23 compared tasks
Task by task
TaskHaiku 5.5 maxOpus 5 mediumSolved by
stet-3169aefaPassed testsPassed testsBoth
stet-89dfbc27Passed testsPassed testsBoth
stet-b28f4fe7Passed testsPassed testsBoth
stet-232b50baFailedPassed testsOpus 5 medium only
stet-5276aa3fPassed testsPassed testsBoth
stet-commit-bbbbae09Passed testsPassed testsBoth
stet-15439c21Passed testsPassed testsBoth
stet-66762b4cFailedPassed testsOpus 5 medium only
stet-commit-5d766ddeEquivalent to the merged fixEquivalent to the merged fixBoth
stet-commit-f0caded9Passed testsPassed testsBoth
stet-43493c52FailedPassed testsOpus 5 medium only
stet-6f84e978Equivalent to the merged fixPassed testsBoth
stet-2450ca2dPassed testsPassed testsBoth
stet-commit-f8ae1177Passed testsFailedHaiku 5.5 max only
stet-commit-342d0ac0Equivalent to the merged fixFailedHaiku 5.5 max only
stet-commit-5e075641FailedFailedNeither
stet-commit-2826765eFailedEquivalent to the merged fixOpus 5 medium only
stet-commit-a8716b88FailedFailedNeither
stet-commit-e928166fFailedFailedNeither
stet-commit-30506f8eFailedFailedNeither
stet-commit-87814583FailedFailedNeither
stet-commit-d2ab0c25FailedFailedNeither
stet-commit-e41c2e2dFailedFailedNeither

Side by side

Side by side
MetricHaiku 5.5 maxOpus 5 medium
Harnessclaude-code—
Reasoning effortmaxmedium
Rank7 of 154 of 15
Stet score57.195% range 45.9–67.663.795% range 53.3–73.6
Solved12/249 passed, 3 judged equivalent14/2412 passed, 2 judged equivalent
Cost / task$3.85$6.78
Agent time / task58.4 min23.2 min

See which setup wins on your repo

Stet replays your merged PRs, scores quality above pass/fail, and delivers comparison reports through your agent.

Run it on your repo

About this data

  • 24 tasks, each a merged change replayed from the repo as it was just before the merge.
  • Data from .
  • Both setups come from the same leaderboard run of 15 setups: bmk_d4ba310fdf50db4ccb2092e3.

See all 15 setups on the leaderboardHow Stet measures