Skip to the comparison

DeepSeek V4.1 Flash (10/17) vs Haiku 5.5

DeepSeek V4.1 Flash (10/17) has a slight quality edge in 1 of 2 pairings; none is a clear lead. 2 pairings; 24 merged changes in Stet-AI/stet. Compare the individual calls below to choose which setup to investigate.

What this comparison is

  • 1 real repositoryStet-AI/stet, private; not public benchmark scores
  • 24 merged changeseach run end to end by the coding agent from the repo as it was before the change
  • Graded against the merged changeby the repo’s own tests and by judges comparing each patch with it
  • Quality, cost and timea smaller sample than public benchmarks, but real agent work on a real codebase

Recorded setup pairings

Each row is a separate recorded comparison. Scores are not pooled or averaged across effort settings. W/D/L means wins, draws and losses; undecided judgments remain undecided.

Stet score: A setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field.

Setup pairingRecorded quality call and recordQuality evidenceCost per taskAgent time per task
DeepSeek V4.1 Flash (10/17) max vs Haiku 5.5 maxDeepSeek V4.1 Flash (10/17) max · codex · max reasoningHaiku 5.5 max · max reasoning21/24 tasks observed by both setups.No clear quality difference: Haiku 5.5 max won 4, DeepSeek V4.1 Flash (10/17) max won 3, 16 tied, 1 undecided; DeepSeek V4.1 Flash (10/17) max leads on cost and agent time: 2.8× cheaper and 1.6× faster.Head to head, Haiku 5.5 max won 4, tied 16 and lost 3 against DeepSeek V4.1 Flash (10/17) max, with 1 undecided.No clear difference
3–4 head to head, 16 tiedDeepSeek V4.1 Flash (10/17) max: 48.3 (95% range 42.9–53.4)Haiku 5.5 max: 48.8 (95% range 38.6–58.7)

DeepSeek V4.1 Flash (10/17) max: $0.28

Haiku 5.5 max: $0.78

DeepSeek V4.1 Flash (10/17) max: 37.2 min

Haiku 5.5 max: 58.4 min

DeepSeek V4.1 Flash (10/17) max vs Haiku 5.5 lowDeepSeek V4.1 Flash (10/17) max · codex · max reasoningHaiku 5.5 low · low reasoning22/24 tasks observed by both setups.DeepSeek V4.1 Flash (10/17) max has a slight quality edge, 3–0 head to head with 21 tied; Haiku 5.5 low leads on cost and agent time: 4.4× cheaper and 3.8× faster.Head to head, DeepSeek V4.1 Flash (10/17) max won 3, tied 21 and lost 0 against Haiku 5.5 low.Slight edge
3–0 head to head, 21 tiedDeepSeek V4.1 Flash (10/17) max: 48.3 (95% range 42.9–53.4)Haiku 5.5 low: 44.8 (95% range 39.0–50.1)

DeepSeek V4.1 Flash (10/17) max: $0.28

Haiku 5.5 low: $0.06

DeepSeek V4.1 Flash (10/17) max: 37.2 min

Haiku 5.5 low: 9.9 min

Questions about these results

Which is better for coding, DeepSeek V4.1 Flash (10/17) or Haiku 5.5?
DeepSeek V4.1 Flash (10/17) has a slight quality edge in 1 of 2 pairings; none is a clear lead. 2 pairings; 24 merged changes in Stet-AI/stet. Compare the individual calls below to choose which setup to investigate.
Where do these coding results come from?
Both coding agents worked the same 24 merged changes from the private Stet-AI/stet repository, each in the real repo as it was before the change. The repo’s tests and judges graded every patch against the merged change; no public benchmark scores are used.
Are reasoning settings combined into one model score?
No. Each effort pairing keeps its own recorded quality call, head-to-head record, cost and agent time. Incompatible scores are never averaged.