DeepSeek V4.1 Flash max vs Grok 4.7 high
Grok 4.7 high leads on speed; DeepSeek V4.1 Flash max on cost; quality is too close to call.
- Quality
- +2.5 pts
- Grok 4.7 high
- 3–1 head to head, 19 tied
- Speed
- 2.6× faster
- Grok 4.7 high
- 18.9 min vs 49.3 min per task
- Cost
- 11.7× cheaper
- DeepSeek V4.1 Flash max
- $0.49 vs $5.76 per task
Stet score
DeepSeek V4.1 Flash maxmax reasoningRank 9 of 17
50.495% range 46.7–54.8
Grok 4.7 highhigh reasoningRank 6 of 17
52.995% range 46.9–58.72.5 points higher
Quality
Head-to-head matchups
DeepSeek V4.1 Flash max won1
Tied19
Grok 4.7 high won3
Task by task
DeepSeek V4.1 Flash max solved 3 tasks that Grok 4.7 high didn’t, and Grok 4.7 high solved 2 that DeepSeek V4.1 Flash max didn’t. Both solved 9; neither solved 10.
stet-43493c52Solved by 12 of 17 setups
- DeepSeek V4.1 Flash maxPassed tests
- Grok 4.7 highFailed
A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change, as in the leaderboard’s Solved column. Stet score also weighs code quality, so it can order the two differently.
All 24 compared tasks
| Task | DeepSeek V4.1 Flash max | Grok 4.7 high | Solved by |
|---|---|---|---|
stet-3169aefa | Passed tests | Passed tests | Both |
stet-89dfbc27 | Passed tests | Passed tests | Both |
stet-b28f4fe7 | Passed tests | Passed tests | Both |
stet-232b50ba | Passed tests | Passed tests | Both |
stet-5276aa3f | Passed tests | Passed tests | Both |
stet-commit-bbbbae09 | Passed tests | Passed tests | Both |
stet-66762b4c | Passed tests | Passed tests | Both |
stet-commit-5d766dde | Equivalent to the merged fix | Equivalent to the merged fix | Both |
stet-43493c52 | Passed tests | Failed | DeepSeek V4.1 Flash max only |
stet-commit-f0caded9 | Failed | Equivalent to the merged fix | Grok 4.7 high only |
stet-commit-f8ae1177 | Passed tests | Passed tests | Both |
stet-15439c21 | Equivalent to the merged fix | Failed | DeepSeek V4.1 Flash max only |
stet-2450ca2d | Failed | Passed tests | Grok 4.7 high only |
stet-commit-342d0ac0 | Equivalent to the merged fix | Failed | DeepSeek V4.1 Flash max only |
stet-6f84e978 | Failed | Failed | Neither |
stet-commit-60a0e6c2 | Failed | Failed | Neither |
stet-commit-5e075641 | Failed | Failed | Neither |
stet-commit-2826765e | Failed | Failed | Neither |
stet-commit-a8716b88 | Failed | Failed | Neither |
stet-commit-e928166f | Failed | Failed | Neither |
stet-commit-30506f8e | Failed | Failed | Neither |
stet-commit-87814583 | Failed | Failed | Neither |
stet-commit-d2ab0c25 | Failed | Failed | Neither |
stet-commit-e41c2e2d | Failed | Failed | Neither |
Side by side
| Metric | DeepSeek V4.1 Flash max | Grok 4.7 high |
|---|---|---|
| Reasoning effort | max | high |
| Rank | 9 of 17 | 6 of 17 |
| Stet score | 50.495% range 46.7–54.8 | 52.995% range 46.9–58.7 |
| Solved | 12/249 passed, 3 judged equivalent | 11/249 passed, 2 judged equivalent |
| Cost / task | $0.49from 23/24 tasks | $5.76 |
| Agent time / task | 49.3 min | 18.9 min |
See which setup wins on your repo
Stet replays your merged PRs, scores quality above pass/fail, and delivers comparison reports through your agent.
Get startedAbout this data
- 24 tasks, each a merged change replayed from the repo as it was just before the merge.
- Data from .
- Both setups come from the same leaderboard run of 17 setups:
bmk_d4ba310fdf50db4ccb2092e3.