Skip to the comparison
STET
Get a free trial

Haiku 5.5 low+guidance vs Haiku 5.5 max

Haiku 5.5 max leads on quality: 16.5 points higher; Haiku 5.5 low+guidance leads on cost and agent time: 13.4× cheaper and 4.7× faster.

24 tasksData from

Stet score

Haiku 5.5 low+guidanceclaude-code · low reasoningRank 10 of 15

42.495% range 33.2–51.7

Haiku 5.5 maxclaude-code · max reasoningRank 6 of 15

58.995% range 47.6–69.416.5 points higher

A directional read: their 95% score ranges overlap.

Cost / task

Haiku 5.5 low+guidance$0.2913.4× cheaper
Haiku 5.5 max$3.85

Agent time / task

Haiku 5.5 low+guidance12.4 min4.7× faster
Haiku 5.5 max58.4 min

Quality

Head-to-head matchups

Haiku 5.5 low+guidance won7

Tied31 undecided

Haiku 5.5 max won13

Haiku 5.5 max’s win share 63%, 95% range 44–80%Ties are within judge noise.

Task by task

Haiku 5.5 low+guidance solved 3 tasks that Haiku 5.5 max didn’t, and Haiku 5.5 max solved 6 that Haiku 5.5 low+guidance didn’t. Both solved 6; neither solved 8.

Haiku 5.5 low+guidance only 3

  • stet-232b50baPassed tests
  • stet-66762b4cPassed tests
  • stet-43493c52Passed tests

Haiku 5.5 max only 6

  • stet-15439c21Passed tests
  • stet-commit-5d766ddeEquivalent to the merged fix
  • stet-6f84e978Equivalent to the merged fix
  • stet-commit-f8ae1177Passed tests
  • stet-2450ca2dPassed tests
  • stet-commit-342d0ac0Equivalent to the merged fix

A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change, as in the leaderboard’s Solved column. Stet score also weighs code quality, so it can order the two differently. 1 of the 24 tasks is left out because one of the two has no scored result on it.

All 23 compared tasks
Task by task
TaskHaiku 5.5 low+guidanceHaiku 5.5 maxSolved by
stet-3169aefaPassed testsPassed testsBoth
stet-89dfbc27Passed testsPassed testsBoth
stet-b28f4fe7Passed testsPassed testsBoth
stet-232b50baPassed testsFailedHaiku 5.5 low+guidance only
stet-5276aa3fPassed testsPassed testsBoth
stet-commit-bbbbae09Passed testsPassed testsBoth
stet-66762b4cPassed testsFailedHaiku 5.5 low+guidance only
stet-15439c21FailedPassed testsHaiku 5.5 max only
stet-commit-f0caded9Equivalent to the merged fixPassed testsBoth
stet-commit-5d766ddeFailedEquivalent to the merged fixHaiku 5.5 max only
stet-43493c52Passed testsFailedHaiku 5.5 low+guidance only
stet-6f84e978FailedEquivalent to the merged fixHaiku 5.5 max only
stet-commit-f8ae1177FailedPassed testsHaiku 5.5 max only
stet-2450ca2dFailedPassed testsHaiku 5.5 max only
stet-commit-342d0ac0FailedEquivalent to the merged fixHaiku 5.5 max only
stet-commit-5e075641FailedFailedNeither
stet-commit-2826765eFailedFailedNeither
stet-commit-a8716b88FailedFailedNeither
stet-commit-30506f8eFailedFailedNeither
stet-commit-87814583FailedFailedNeither
stet-commit-d2ab0c25FailedFailedNeither
stet-commit-e41c2e2dFailedFailedNeither
stet-commit-e928166fFailedFailedNeither

Side by side

Side by side
MetricHaiku 5.5 low+guidanceHaiku 5.5 max
Harnessclaude-codeclaude-code
Reasoning effortlowmax
Rank10 of 156 of 15
Stet score42.495% range 33.2–51.758.995% range 47.6–69.4
Solved9/248 passed, 1 judged equivalent12/249 passed, 3 judged equivalent
Cost / task$0.29$3.85
Agent time / task12.4 min58.4 min

See which setup wins on your repo

Stet replays your merged PRs, scores quality above pass/fail, and delivers comparison reports through your agent.

Run it on your repo

About this data

  • 24 tasks, each a merged change replayed from the repo as it was just before the merge.
  • Data from .
  • Both setups come from the same leaderboard run of 15 setups: bmk_d4ba310fdf50db4ccb2092e3.

See all 15 setups on the leaderboardHow Stet measures