DeepSeek V4.1 Flash (10/17) vs GPT 5.6 Luna
GPT 5.6 Luna faster in both pairings. DeepSeek V4.1 Flash (10/17) has a slight quality edge in 1 of 2 pairings; none is a clear lead. 2 pairings; 24 merged changes in Stet-AI/stet. Compare the individual calls below to choose which setup to investigate.
What this comparison is
- 1 real repositoryStet-AI/stet, private; not public benchmark scores
- 24 merged changeseach run end to end by the coding agent from the repo as it was before the change
- Graded against the merged changeby the repo’s own tests and by judges comparing each patch with it
- Quality, cost and timea smaller sample than public benchmarks, but real agent work on a real codebase
Compare coding modelsLow vs high vs xhigh/max reasoning for coding agents
Recorded setup pairings
Each row is a separate recorded comparison. Scores are not pooled or averaged across effort settings. W/D/L means wins, draws and losses; undecided judgments remain undecided.
Stet score: A setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field.
| Setup pairing | Recorded quality call and record | Quality evidence | Cost per task | Agent time per task |
|---|---|---|---|---|
| DeepSeek V4.1 Flash (10/17) max vs GPT 5.6 Luna maxDeepSeek V4.1 Flash (10/17) max · codex · max reasoningGPT 5.6 Luna max · max reasoning22/24 tasks observed by both setups. | No clear quality difference: DeepSeek V4.1 Flash (10/17) max won 3, GPT 5.6 Luna max won 2, 19 tied; DeepSeek V4.1 Flash (10/17) max leads on cost: 3.2× cheaper; GPT 5.6 Luna max leads on agent time: 1.3× faster.Head to head, DeepSeek V4.1 Flash (10/17) max won 3, tied 19 and lost 2 against GPT 5.6 Luna max. | No clear difference 3–2 head to head, 19 tiedDeepSeek V4.1 Flash (10/17) max: 48.3 (95% range 42.9–53.4)GPT 5.6 Luna max: 46.7 (95% range 40.8–52.4) | DeepSeek V4.1 Flash (10/17) max: $0.28 GPT 5.6 Luna max: $0.90 | DeepSeek V4.1 Flash (10/17) max: 37.2 min GPT 5.6 Luna max: 28.5 min |
| DeepSeek V4.1 Flash (10/17) max vs GPT 5.6 Luna lowDeepSeek V4.1 Flash (10/17) max · codex · max reasoningGPT 5.6 Luna low · codex · low reasoning22/24 tasks observed by both setups. | DeepSeek V4.1 Flash (10/17) max has a slight quality edge, 3–1 head to head with 20 tied; GPT 5.6 Luna low leads on cost and agent time: 5.5× cheaper and 5.6× faster.Head to head, DeepSeek V4.1 Flash (10/17) max won 3, tied 20 and lost 1 against GPT 5.6 Luna low. | Slight edge 3–1 head to head, 20 tiedDeepSeek V4.1 Flash (10/17) max: 48.3 (95% range 42.9–53.4)GPT 5.6 Luna low: 43.4 (95% range 36.6–49.9) | DeepSeek V4.1 Flash (10/17) max: $0.28 GPT 5.6 Luna low: $0.05 | DeepSeek V4.1 Flash (10/17) max: 37.2 min GPT 5.6 Luna low: 6.7 min |
Questions about these results
- Which is better for coding, DeepSeek V4.1 Flash (10/17) or GPT 5.6 Luna?
- GPT 5.6 Luna faster in both pairings. DeepSeek V4.1 Flash (10/17) has a slight quality edge in 1 of 2 pairings; none is a clear lead. 2 pairings; 24 merged changes in Stet-AI/stet. Compare the individual calls below to choose which setup to investigate.
- Where do these coding results come from?
- Both coding agents worked the same 24 merged changes from the private Stet-AI/stet repository, each in the real repo as it was before the change. The repo’s tests and judges graded every patch against the merged change; no public benchmark scores are used.
- Are reasoning settings combined into one model score?
- No. Each effort pairing keeps its own recorded quality call, head-to-head record, cost and agent time. Incompatible scores are never averaged.