Skip to the comparison

Haiku 5.5 vs SWE-2

Haiku 5.5 faster in both pairings. No clear quality difference in any pairing.

2 pairings; 24 merged changes in Stet-AI/stet. Data from

Recorded setup pairings

Haiku 5.5 max vs SWE-2 max

Haiku 5.5 max leads on speed; SWE-2 max on cost; no clear quality difference.

23/24 tasks run by both setups

Quality
Even
No clear difference
1–6 head to head, 16 tied, 1 undecided
Speed
1.2× faster
Haiku 5.5 max
58.4 min vs 68.3 min per task
Cost
1.4× cheaper
SWE-2 max
$2.76 vs $3.83 per task; $2.76 from 23/24 tasks

Stet score

Haiku 5.5 maxclaude-code · max reasoningRank 12 of 18

48.095% range 38.3–57.4

SWE-2 maxdevin · max reasoningRank 1 of 18

62.395% range 53.4–71.6

Their 95% score ranges overlap.

Head-to-head matchups

Haiku 5.5 max won1

Tied16

1 undecided

SWE-2 max won6

SWE-2 max’s win share 61%, 95% range 50–72%Ties are within judge noise.

Haiku 5.5 low vs SWE-2 max

Haiku 5.5 low leads on speed and cost; no clear quality difference.

Quality
Even
No clear difference
1–11 head to head, 12 tied
Speed
6.9× faster
Haiku 5.5 low
9.9 min vs 68.3 min per task
Cost
11× cheaper
Haiku 5.5 low
$0.25 vs $2.76 per task; $2.76 from 23/24 tasks

Stet score

Haiku 5.5 lowclaude-code · low reasoningRank 17 of 18

43.795% range 38.6–48.3

SWE-2 maxdevin · max reasoningRank 1 of 18

62.395% range 53.4–71.6

Their 95% score ranges don’t overlap.

Head-to-head matchups

Haiku 5.5 low won1

Tied12

SWE-2 max won11

SWE-2 max’s win share 71%, 95% range 58–81%Ties are within judge noise.

Questions about these results

Which is better for coding, Haiku 5.5 or SWE-2?
Haiku 5.5 faster in both pairings. No clear quality difference in any pairing. 2 pairings; 24 merged changes in Stet-AI/stet.
Where do these coding results come from?
Both coding agents worked the same 24 merged changes from the private Stet-AI/stet repository, each in the real repo as it was before the change. The repo’s tests and judges graded every patch against the merged change; no public benchmark scores are used.
Are reasoning settings combined into one model score?
No. Each effort pairing keeps its own recorded quality call, head-to-head record, cost and agent time. Incompatible scores are never averaged.