Skip to the comparison
STET
Run it on your repo

GPT 5.6 Luna max vs GPT 6 Luna low

GPT 6 Luna low leads on speed and cost; quality is too close to call.

Quality
+9.6 pts
GPT 5.6 Luna max
5–1 head to head, 17 tied
Speed
4.1× faster
GPT 6 Luna low
6.9 min vs 28.5 min per task
Cost
16.8× cheaper
GPT 6 Luna low
$0.05 vs $0.90 per task

24 tasksData from

Stet score

GPT 5.6 Luna maxmax reasoningRank 9 of 17

49.995% range 44.5–55.69.6 points higher

GPT 6 Luna lowlow reasoningRank 16 of 17

40.395% range 30.6–49.8

A directional read: their 95% score ranges overlap.

Quality

Head-to-head matchups

GPT 5.6 Luna max won5

Tied17

GPT 6 Luna low won1

1 undecided
GPT 5.6 Luna max’s win share 59%, 95% range 50–69%Ties are within judge noise.

Task by task

GPT 5.6 Luna max solved 6 tasks that GPT 6 Luna low didn’t, and GPT 6 Luna low solved 1 that GPT 5.6 Luna max didn’t. Both solved 4; neither solved 12.

stet-5276aa3fSolved by 15 of 17 setups

  • GPT 5.6 Luna maxPassed tests
  • GPT 6 Luna lowFailed

A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change, as in the leaderboard’s Solved column. Stet score also weighs code quality, so it can order the two differently. 1 of the 24 tasks is left out because one of the two has no scored result on it.

All 23 compared tasks
Task by task
TaskGPT 5.6 Luna maxGPT 6 Luna lowSolved by
stet-232b50baPassed testsPassed testsBoth
stet-3169aefaPassed testsPassed testsBoth
stet-5276aa3fPassed testsFailedGPT 5.6 Luna max only
stet-89dfbc27Passed testsPassed testsBoth
stet-b28f4fe7Passed testsPassed testsBoth
stet-commit-bbbbae09Passed testsNo change madeGPT 5.6 Luna max only
stet-66762b4cEquivalent to the merged fixFailedGPT 5.6 Luna max only
stet-commit-5d766ddeEquivalent to the merged fixNo change madeGPT 5.6 Luna max only
stet-commit-f0caded9Equivalent to the merged fixFailedGPT 5.6 Luna max only
stet-43493c52Passed testsFailedGPT 5.6 Luna max only
stet-commit-f8ae1177FailedNo change madeNeither
stet-15439c21FailedPassed testsGPT 6 Luna low only
stet-2450ca2dFailedFailedNeither
stet-commit-342d0ac0FailedFailedNeither
stet-6f84e978FailedFailedNeither
stet-commit-5e075641FailedFailedNeither
stet-commit-2826765eFailedFailedNeither
stet-commit-a8716b88FailedFailedNeither
stet-commit-e928166fFailedFailedNeither
stet-commit-30506f8eFailedFailedNeither
stet-commit-87814583FailedFailedNeither
stet-commit-d2ab0c25FailedFailedNeither
stet-commit-e41c2e2dFailedFailedNeither

Side by side

Side by side
MetricGPT 5.6 Luna maxGPT 6 Luna low
Reasoning effortmaxlow
Rank9 of 1716 of 17
Stet score49.995% range 44.5–55.640.395% range 30.6–49.8
Solved10/247 passed, 3 judged equivalent5/245 passed, 0 judged equivalent
Cost / task$0.90$0.05
Agent time / task28.5 min6.9 min

See which setup wins on your repo

Stet replays your merged PRs, scores quality above pass/fail, and delivers comparison reports through your agent.

Run it on your repo

About this data

  • 24 tasks, each a merged change replayed from the repo as it was just before the merge.
  • Data from .
  • Both setups come from the same leaderboard run of 17 setups: bmk_d4ba310fdf50db4ccb2092e3.

See all 17 setups on the leaderboardHow Stet measures