Skip to the comparison

Pareto 26.9 medium vs SWE-2 Max high

Pareto 26.9 medium leads on speed; quality is too close to call.

Quality
+2.8 pts
SWE-2 Max high
4–3 head to head, 16 tied
Speed
1.7× faster
Pareto 26.9 medium
39.2 min vs 68.3 min per task

24 tasksData from Cost not recorded for SWE-2 Max high

Stet score

Pareto 26.9 mediummedium reasoningRank 3 of 17

55.295% range 50.5–60.5

SWE-2 Max highhigh reasoningRank 2 of 17

58.095% range 49.7–66.82.8 points higher

A directional read: their 95% score ranges overlap.

Quality

Head-to-head matchups

Pareto 26.9 medium won3

Tied16

SWE-2 Max high won4

1 undecided
SWE-2 Max high’s win share 53%, 95% range 42–64%Ties are within judge noise.

Task by task

Pareto 26.9 medium solved 2 tasks that SWE-2 Max high didn’t, and SWE-2 Max high solved 7 that Pareto 26.9 medium didn’t. Both solved 9; neither solved 6.

stet-commit-5d766ddeSolved by 14 of 17 setups

  • Pareto 26.9 mediumFailed
  • SWE-2 Max highEquivalent to the merged fix

A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change, as in the leaderboard’s Solved column. Stet score also weighs code quality, so it can order the two differently.

All 24 compared tasks
Task by task
TaskPareto 26.9 mediumSWE-2 Max highSolved by
stet-3169aefaPassed testsPassed testsBoth
stet-89dfbc27Passed testsPassed testsBoth
stet-b28f4fe7Passed testsPassed testsBoth
stet-232b50baPassed testsPassed testsBoth
stet-5276aa3fPassed testsPassed testsBoth
stet-commit-bbbbae09Passed testsPassed testsBoth
stet-66762b4cPassed testsPassed testsBoth
stet-commit-5d766ddeFailedEquivalent to the merged fixSWE-2 Max high only
stet-43493c52Passed testsFailedPareto 26.9 medium only
stet-commit-f0caded9Equivalent to the merged fixPassed testsBoth
stet-commit-f8ae1177Passed testsFailedPareto 26.9 medium only
stet-15439c21FailedEquivalent to the merged fixSWE-2 Max high only
stet-2450ca2dPassed testsPassed testsBoth
stet-commit-342d0ac0FailedEquivalent to the merged fixSWE-2 Max high only
stet-6f84e978FailedPassed testsSWE-2 Max high only
stet-commit-60a0e6c2FailedPassed testsSWE-2 Max high only
stet-commit-5e075641FailedPassed testsSWE-2 Max high only
stet-commit-2826765eFailedFailedNeither
stet-commit-a8716b88FailedEquivalent to the merged fixSWE-2 Max high only
stet-commit-e928166fFailedFailedNeither
stet-commit-30506f8eFailedFailedNeither
stet-commit-87814583FailedFailedNeither
stet-commit-d2ab0c25FailedFailedNeither
stet-commit-e41c2e2dFailedFailedNeither

Side by side

Side by side
MetricPareto 26.9 mediumSWE-2 Max high
Reasoning effortmediumhigh
Rank3 of 172 of 17
Stet score55.295% range 50.5–60.558.095% range 49.7–66.8
Solved11/2410 passed, 1 judged equivalent16/2412 passed, 4 judged equivalent
Cost / task$5.98not recorded
Agent time / task39.2 min68.3 min

See which setup wins on your repo

Stet replays your merged PRs, scores quality above pass/fail, and delivers comparison reports through your agent.

Get started