Skip to the comparison

DeepSeek V4.1 Flash (10/17) max vs Muse Wonder Full low

Muse Wonder Full low leads on quality and speed; DeepSeek V4.1 Flash (10/17) max on cost.

Quality
+8.2 pts
Muse Wonder Full low
Directional lead at this sample size
3–0 head to head, 21 tied
Speed
1.7× faster
Muse Wonder Full low
21.6 min vs 37.2 min per task
Cost
10.1× cheaper
DeepSeek V4.1 Flash (10/17) max
$0.28 vs $2.85 per task

What this comparison is

  • 1 real repositoryStet-AI/stet, private; not public benchmark scores
  • 24 merged changeseach run end to end by the coding agent from the repo as it was before the change
  • Graded against the merged changeby the repo’s own tests and by judges comparing each patch with it
  • Quality, cost and timea smaller sample than public benchmarks, but real agent work on a real codebase

24 tasksData from

Stet score

DeepSeek V4.1 Flash (10/17) maxcodex · max reasoningRank 11 of 18

48.395% range 42.9–53.4

Muse Wonder Full lowlow reasoningRank 3 of 18

56.595% range 50.8–63.08.2 points higher

A directional read: their 95% score ranges overlap.

Quality

Head-to-head matchups

DeepSeek V4.1 Flash (10/17) max won0

Tied21

Muse Wonder Full low won3

Muse Wonder Full low’s win share 56%, 95% range 50–64%Ties are within judge noise.

Task by task

DeepSeek V4.1 Flash (10/17) max solved 0 tasks that Muse Wonder Full low didn’t, and Muse Wonder Full low solved 3 that DeepSeek V4.1 Flash (10/17) max didn’t. Both solved 10; neither solved 9.

On the 10 tasks both solved, judges preferred Muse Wonder Full low’s patch on 1 and called 9 even.

0Only DeepSeek V4.1 Flash (10/17) max solved

None

3Only Muse Wonder Full low solved

  • 113 changed lines in 4 filesstet-commit-f0caded9
  • 149 changed lines in 2 filesstet-commit-e928166f
  • 1,312 changed lines in 9 filesstet-commit-f8ae1177Judges preferred Muse Wonder Full low’s patch

10Both solved

  • 2 changed lines in 1 filestet-2450ca2d
  • 4 changed lines in 1 filestet-43493c52
  • 6 changed lines in 1 filestet-3169aefa
  • 9 changed lines in 1 filestet-89dfbc27
  • 10 changed lines in 1 filestet-232b50ba
  • 23 changed lines in 2 filesstet-5276aa3f
  • 29 changed lines in 1 filestet-b28f4fe7
  • 84 changed lines in 1 filestet-66762b4c
  • 218 changed lines in 2 filesstet-commit-5d766dde
  • 941 changed lines in 7 filesstet-commit-bbbbae09Judges preferred Muse Wonder Full low’s patch

9Neither solved

  • 476 changed lines in 17 filesstet-commit-30506f8e
  • 481 changed lines in 3 filesstet-commit-5e075641
  • 552 changed lines in 11 filesstet-commit-60a0e6c2
  • 564 changed lines in 10 filesstet-commit-a8716b88
  • 640 changed lines in 8 filesstet-commit-342d0ac0
  • 763 changed lines in 8 filesstet-commit-87814583
  • 977 changed lines in 5 filesstet-commit-e41c2e2d
  • 1,236 changed lines in 16 filesstet-commit-d2ab0c25
  • 1,420 changed lines in 19 filesstet-commit-2826765e

Bar: changed lines in the merged change, log scale

A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change, as in the leaderboard’s Solved column. Stet score also weighs code quality, so it can order the two differently. 2 of the 24 tasks are left out because one of the two has no scored result on them.

All 22 compared tasks
Task by task
TaskDeepSeek V4.1 Flash (10/17) maxMuse Wonder Full lowSolved byBetter patch
stet-3169aefa · 6 changed lines in 1 filePassed tests$0.06 · 15.9 minPassed tests$0.94 · 4.9 minBothEven
stet-89dfbc27 · 9 changed lines in 1 filePassed tests$0.05 · 4.9 minPassed tests$5.57 · 57.3 minBothEven
stet-b28f4fe7 · 29 changed lines in 1 filePassed tests$0.06 · 28.7 minPassed tests$2.25 · 28.1 minBothEven
stet-232b50ba · 10 changed lines in 1 filePassed tests$0.05 · 28.0 minPassed tests$0.70 · 13.3 minBothEven
stet-5276aa3f · 23 changed lines in 2 filesPassed tests$0.02 · 14.4 minPassed tests$1.13 · 21.4 minBothEven
stet-commit-bbbbae09 · 941 changed lines in 7 filesPassed tests$0.65 · 47.7 minPassed tests$5.62 · 35.1 minBothMuse Wonder Full low
stet-66762b4c · 84 changed lines in 1 filePassed tests$0.08 · 9.1 minPassed tests$0.74 · 6.5 minBothEven
stet-commit-5d766dde · 218 changed lines in 2 filesEquivalent to the merged fix$0.09 · 18.5 minEquivalent to the merged fix$1.23 · 5.7 minBothEven
stet-43493c52 · 4 changed lines in 1 filePassed tests$0.14 · 16.0 minPassed tests$0.97 · 6.4 minBothEven
stet-commit-f0caded9 · 113 changed lines in 4 filesFailed$0.09 · 14.6 minEquivalent to the merged fix$2.03 · 13.4 minMuse Wonder Full low onlyEven
stet-commit-f8ae1177 · 1,312 changed lines in 9 filesFailed$0.54 · 75.0 minPassed tests$4.65 · 42.6 minMuse Wonder Full low onlyMuse Wonder Full low
stet-2450ca2d · 2 changed lines in 1 filePassed tests$0.07 · 10.1 minPassed tests$1.04 · 10.0 minBothEven
stet-commit-342d0ac0 · 640 changed lines in 8 filesFailed$0.20 · 38.3 minFailed$3.34 · 12.2 minNeitherEven
stet-commit-60a0e6c2 · 552 changed lines in 11 filesFailed$0.34 · 39.2 minFailed$4.83 · 14.1 minNeitherEven
stet-commit-5e075641 · 481 changed lines in 3 filesFailed$0.46 · 53.1 minFailed$3.16 · 17.6 minNeitherEven
stet-commit-2826765e · 1,420 changed lines in 19 filesFailed$0.56 · 50.2 minFailed$4.30 · 16.6 minNeitherEven
stet-commit-a8716b88 · 564 changed lines in 10 filesFailed$0.30 · 40.2 minFailed$3.01 · 10.4 minNeitherEven
stet-commit-e928166f · 149 changed lines in 2 filesFailed$0.29 · 42.0 minEquivalent to the merged fix$2.00 · 10.5 minMuse Wonder Full low onlyEven
stet-commit-30506f8e · 476 changed lines in 17 filesFailed$0.38 · 48.8 minFailed$4.14 · 25.9 minNeitherEven
stet-commit-87814583 · 763 changed lines in 8 filesFailed$0.50 · 62.2 minFailed$3.68 · 13.5 minNeitherEven
stet-commit-d2ab0c25 · 1,236 changed lines in 16 filesFailed$0.71 · 76.4 minFailed$5.51 · 41.6 minNeitherEven
stet-commit-e41c2e2d · 977 changed lines in 5 filesFailed$0.96 · 110.4 minFailed$5.42 · 76.5 minNeitherEven

Cost and time on each task

Each dot is one task, on a log scale: filled where the setup solved it, hollow where it didn’t. The line marks the median.

Cost per task

DeepSeek V4.1 Flash (10/17) maxmean $0.30 · median $0.25

Muse Wonder Full lowmean $3.01 · median $3.09

DeepSeek V4.1 Flash (10/17) max cost less on all 22 tasks.

Agent time per task

DeepSeek V4.1 Flash (10/17) maxmean 38.3 min · median 38.7 min

Muse Wonder Full lowmean 22.0 min · median 13.8 min

Muse Wonder Full low finished faster on 20 of 22 tasks; DeepSeek V4.1 Flash (10/17) max on 2.

Where each setup is the better choice

The leaderboard’s readings on groups of tasks. Each group holds fewer tasks than the overall standings, so treat these as leads to test on your repo.

Small fixes

Cheaper here without solving fewer: DeepSeek V4.1 Flash (10/17) max

10 tasks · 2–84 changed lines in 1–2 files

DeepSeek V4.1 Flash (10/17) max solved as many, 8 of 8, at 24.9× lower cost per task.

DeepSeek V4.1 Flash (10/17) max was 1.2× faster.

Small fixes
DeepSeek V4.1 Flash (10/17) maxMuse Wonder Full low
Solved8/88/8
Cost / task$0.07$1.67
Agent time / task15.9 min18.5 min

Large, cross-cutting changes

14 tasks · 113–1,420 changed lines in up to 19 files

Muse Wonder Full low solved 3 more, 5 of 14 to 2; DeepSeek V4.1 Flash (10/17) max cost 8.7× less per task.

Muse Wonder Full low was 2.1× faster.

Large, cross-cutting changes
DeepSeek V4.1 Flash (10/17) maxMuse Wonder Full low
Solved2/145/14
Cost / task$0.43$3.78
Agent time / task51.2 min24.0 min

Side by side

Side by side
MetricDeepSeek V4.1 Flash (10/17) maxMuse Wonder Full low
Harnesscodex—
Reasoning effortmaxlow
Rank11 of 183 of 18
Stet score48.395% range 42.9–53.456.595% range 50.8–63.0
Solved10/249 passed, 1 judged equivalent14/2411 passed, 3 judged equivalent
Cost / task$0.28$2.85
Agent time / task37.2 min21.6 min

Questions

Which is better for coding, DeepSeek V4.1 Flash (10/17) max or Muse Wonder Full low?
Muse Wonder Full low leads on quality, 3–0 head to head with 21 tied (directional at this sample size); DeepSeek V4.1 Flash (10/17) max leads on cost: 10.1× cheaper; Muse Wonder Full low leads on agent time: 1.7× faster. Measured on 24 merged changes from Stet-AI/stet.
How were DeepSeek V4.1 Flash (10/17) max and Muse Wonder Full low tested?
Both coding agents worked the same 24 merged changes from the private Stet-AI/stet repository, each in the real repo as it was before the change. The repo’s tests and judges graded every patch against the merged change; no public benchmark scores are used. Stet score: A setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field.

See which setup wins on your repo

Stet replays your merged changes, scores quality above pass/fail, and delivers comparison reports through your agent.

Get started