# Pareto 26.9 medium vs SWE-2 Max high: real-repository coding comparison

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/leaderboard/compare/pareto-26-9-medium-vs-swe-2-max-high
Source observed: 2026-11-19T11:08:40.117Z
Last checked: 2026-11-19T11:08:40.117+00:00
Indexed: 2026-11-19T11:14:27.054+00:00

---

# Pareto 26.9 medium vs SWE-2 Max high

We compare two coding-agent setups on real work from our private `Stet-AI/stet` repository. Pareto 26.9 medium used `claude-code` with medium reasoning; SWE-2 Max high used `devin` with high reasoning. The results compare complete setups, including their harnesses and reasoning settings, rather than models alone.

## At a glance

- **Quality:** No clear difference. SWE-2 Max high won 8 head-to-head tasks, Pareto 26.9 medium won 3, and 13 were tied. Ties are within judge noise.
- **Speed:** Pareto 26.9 medium was **1.7× faster**, averaging 39.2 minutes per task versus 68.3 minutes.
- **Cost:** Pareto 26.9 medium averaged **$5.98 per task**. SWE-2 Max high cost was not recorded.

## What we compared

- **One real repository:** private `Stet-AI/stet`; these are not public benchmark scores.
- **24 merged changes:** each setup ran every change end to end from the repository state immediately before that change.
- **Grading against the merged change:** we used the repository’s tests and judges comparing each patch with the merged change.
- **Quality, cost, and time:** this is a smaller sample than public benchmarks, focused on real agent work in a real codebase.

Data covers 24 tasks from **November 19, 2026**. Cost was not recorded for SWE-2 Max high.

## Stet score and leaderboard position

The Stet score is a setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field.

| Setup | Harness and reasoning | Rank | Stet score |
| --- | --- | ---: | --- |
| Pareto 26.9 medium | `claude-code`; medium reasoning | 7 of 18 | 52.3; 95% range 44.9–59.5 |
| SWE-2 Max high | `devin`; high reasoning | 1 of 18 | 63.8; 95% range 54.9–72.7 |

The 95% score ranges overlap.

## Quality

### Head-to-head matchups

- Pareto 26.9 medium won **3**.
- **13** were tied.
- SWE-2 Max high won **8**.
- SWE-2 Max high’s win share was **61%**, with a 95% range of **48–74%**.

### Other quality measures

| Metric | Pareto 26.9 medium | SWE-2 Max high |
| --- | --- | --- |
| Code review pass rate | 35% — 23 tasks | 33% — 24 tasks |
| Checklist score | 0.80 — 24 tasks | 0.84 — 24 tasks |
| Equivalence rate | 39% — 23 tasks | 54% — 24 tasks |

These measures are not part of the headline Stet score and have no confidence interval. A small gap may be noise; the higher recorded value is not a proven lead.

## Task-by-task results

Pareto 26.9 medium solved 2 tasks that SWE-2 Max high did not. SWE-2 Max high solved 7 that Pareto 26.9 medium did not. Both solved 9, and neither solved 6. Among the 9 tasks both solved, judges preferred Pareto’s patch on 1, SWE-2 Max high’s patch on 1, and called 7 even.

### Only Pareto 26.9 medium solved — 2

- `stet-43493c52` — 4 changed lines in 1 file; judges preferred Pareto’s patch.
- `stet-commit-f8ae1177` — 1,312 changed lines in 9 files; judges preferred Pareto’s patch.

### Only SWE-2 Max high solved — 7

- `stet-15439c21` — 5 changed lines in 2 files; judges preferred SWE-2 Max high’s patch.
- `stet-6f84e978` — 23 changed lines in 1 file; judges preferred SWE-2 Max high’s patch.
- `stet-commit-5d766dde` — 218 changed lines in 2 files; judges preferred SWE-2 Max high’s patch.
- `stet-commit-5e075641` — 481 changed lines in 3 files; judges preferred SWE-2 Max high’s patch.
- `stet-commit-60a0e6c2` — 552 changed lines in 11 files; judges preferred SWE-2 Max high’s patch.
- `stet-commit-a8716b88` — 564 changed lines in 10 files; judges preferred SWE-2 Max high’s patch.
- `stet-commit-342d0ac0` — 640 changed lines in 8 files; judges preferred SWE-2 Max high’s patch.

### Both solved — 9

- `stet-2450ca2d` — 2 changed lines in 1 file.
- `stet-3169aefa` — 6 changed lines in 1 file.
- `stet-89dfbc27` — 9 changed lines in 1 file.
- `stet-232b50ba` — 10 changed lines in 1 file.
- `stet-5276aa3f` — 23 changed lines in 2 files.
- `stet-b28f4fe7` — 29 changed lines in 1 file.
- `stet-66762b4c` — 84 changed lines in 1 file.
- `stet-commit-f0caded9` — 113 changed lines in 4 files; judges preferred SWE-2 Max high’s patch.
- `stet-commit-bbbbae09` — 941 changed lines in 7 files; judges preferred Pareto’s patch.

### Neither solved — 6

- `stet-commit-e928166f` — 149 changed lines in 2 files.
- `stet-commit-30506f8e` — 476 changed lines in 17 files.
- `stet-commit-87814583` — 763 changed lines in 8 files.
- `stet-commit-e41c2e2d` — 977 changed lines in 5 files.
- `stet-commit-d2ab0c25` — 1,236 changed lines in 16 files.
- `stet-commit-2826765e` — 1,420 changed lines in 19 files.

A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change. Our Stet score also weighs code quality, so it can order the setups differently from the Solved count. The bar represents changed lines on a logarithmic scale.

## All 24 compared tasks

| Task | Pareto 26.9 medium | SWE-2 Max high | Solved by | Better patch |
| --- | --- | --- | --- | --- |
| `stet-89dfbc27` · 9 lines, 1 file | Passed tests; $3.94 · 47.2 min | Passed tests; 4.5 min | Both | Even |
| `stet-b28f4fe7` · 29 lines, 1 file | Passed tests; $3.17 · 22.7 min | Passed tests; 16.7 min | Both | Even |
| `stet-232b50ba` · 10 lines, 1 file | Passed tests; $3.09 · 42.8 min | Passed tests; 11.9 min | Both | Even |
| `stet-3169aefa` · 6 lines, 1 file | Passed tests; $2.42 · 14.8 min | Passed tests; 5.3 min | Both | Even |
| `stet-5276aa3f` · 23 lines, 2 files | Passed tests; $0.69 · 13.0 min | Passed tests; 10.7 min | Both | Even |
| `stet-66762b4c` · 84 lines, 1 file | Passed tests; $3.39 · 50.6 min | Passed tests; 11.2 min | Both | Even |
| `stet-commit-bbbbae09` · 941 lines, 7 files | Passed tests; $14.28 · 95.4 min | Passed tests; 89.7 min | Both | Pareto 26.9 medium |
| `stet-commit-5d766dde` · 218 lines, 2 files | Failed; $1.02 · 7.0 min | Equivalent to merged fix; 20.0 min | SWE-2 Max high only | SWE-2 Max high |
| `stet-43493c52` · 4 lines, 1 file | Passed tests; $2.04 · 29.5 min | Failed; 35.3 min | Pareto only | Pareto |
| `stet-commit-f0caded9` · 113 lines, 4 files | Equivalent to merged fix; $8.97 · 60.8 min | Passed tests; 22.6 min | Both | SWE-2 Max high |
| `stet-15439c21` · 5 lines, 2 files | Failed; $2.73 · 36.0 min | Passed tests; 34.3 min | SWE-2 Max high only | SWE-2 Max high |
| `stet-commit-f8ae1177` · 1,312 lines, 9 files | Passed tests; $9.84 · 52.6 min | Failed; 300.0 min | Pareto only | Pareto |
| `stet-2450ca2d` · 2 lines, 1 file | Passed tests; $0.61 · 12.3 min | Passed tests; 11.8 min | Both | Even |
| `stet-commit-342d0ac0` · 640 lines, 8 files | Failed; $7.95 · 42.6 min | Equivalent to merged fix; 146.4 min | SWE-2 Max high only | SWE-2 Max high |
| `stet-6f84e978` · 23 lines, 1 file | Failed; $2.15 · 20.9 min | Passed tests; 42.4 min | SWE-2 Max high only | SWE-2 Max high |
| `stet-commit-5e075641` · 481 lines, 3 files | Failed; $2.54 · 23.6 min | Passed tests; 86.0 min | SWE-2 Max high only | SWE-2 Max high |
| `stet-commit-60a0e6c2` · 552 lines, 11 files | Failed; $5.64 · 20.5 min | Passed tests; 106.1 min | SWE-2 Max high only | SWE-2 Max high |
| `stet-commit-2826765e` · 1,420 lines, 19 files | Failed; $8.80 · 40.3 min | Failed; 144.5 min | Neither | Even |
| `stet-commit-a8716b88` · 564 lines, 10 files | Failed; $13.86 · 42.7 min | Equivalent to merged fix; 102.3 min | SWE-2 Max high only | SWE-2 Max high |
| `stet-commit-e928166f` · 149 lines, 2 files | Failed; $7.09 · 34.0 min | Failed; 59.4 min | Neither | Even |
| `stet-commit-30506f8e` · 476 lines, 17 files | Failed; $16.87 · 66.0 min | Failed; 164.8 min | Neither | Even |
| `stet-commit-87814583` · 763 lines, 8 files | Failed; $11.99 · 89.0 min | Failed; 73.6 min | Neither | Even |
| `stet-commit-d2ab0c25` · 1,236 lines, 16 files | Failed; $1.11 · 8.3 min | Failed; 68.9 min | Neither | Even |
| `stet-commit-e41c2e2d` · 977 lines, 5 files | Failed; $9.40 · 67.9 min | Failed; 69.8 min | Neither | Even |

## Cost and time

We show one dot per task on a logarithmic scale; filled dots indicate a solved task, hollow dots an unsolved task, and the line marks the median.

| Metric | Pareto 26.9 medium | SWE-2 Max high |
| --- | --- | --- |
| Cost per task | Mean **$5.98**; median **$3.66** | Not recorded |
| Agent time per task | Mean **39.2 min**; median **38.2 min** | Mean **68.3 min**; median **50.9 min** |

Pareto finished faster on 13 of 24 tasks; SWE-2 Max high finished faster on 11.

## Where each setup is the better choice

These are readings on smaller task groups, so use them as leads to test on your repository.

### Small fixes

10 tasks, with 2–84 changed lines in 1–2 files.

SWE-2 Max high solved 9/10 versus Pareto’s 8/10 and was 1.6× faster.

| Metric | Pareto 26.9 medium | SWE-2 Max high |
| --- | ---: | ---: |
| Solved | 8/10 | 9/10 |
| Cost per task | $2.42 | Not recorded |
| Agent time per task | 29.0 min | 18.4 min |

### Large, cross-cutting changes

14 tasks, with 113–1,420 changed lines in up to 19 files.

SWE-2 Max high solved 7/14 versus Pareto’s 3/14, while Pareto was 2.2× faster.

| Metric | Pareto 26.9 medium | SWE-2 Max high |
| --- | ---: | ---: |
| Solved | 3/14 | 7/14 |
| Cost per task | $8.53 | Not recorded |
| Agent time per task | 46.5 min | 103.9 min |

## Side-by-side summary

| Metric | Pareto 26.9 medium | SWE-2 Max high |
| --- | --- | --- |
| Harness | `claude-code` | `devin` |
| Reasoning effort | Medium | High |
| Rank | 7 of 18 | 1 of 18 |
| Stet score | 52.3; 95% range 44.9–59.5 | 63.8; 95% range 54.9–72.7 |
| Solved | 11/24 — 10 passed, 1 judged equivalent | 16/24 — 13 passed, 3 judged equivalent |
| Cost per task | $5.98 | Not recorded |
| Agent time per task | 39.2 min | 68.3 min |

## Questions

### Which is better for coding?

We found no clear quality difference: SWE-2 Max high won 8, Pareto 26.9 medium won 3, and 13 were tied. Pareto 26.9 medium led on agent time at 1.7× faster. SWE-2 Max high’s cost was not recorded. The comparison used 24 merged changes from `Stet-AI/stet`.

### How did we test them?

Both coding agents worked on the same 24 merged changes in the private `Stet-AI/stet` repository, starting from the real repository state before each change. The repository’s tests and judges graded every patch against the merged change. We did not use public benchmark scores.

## Test your repository

We replay your merged changes, score quality beyond pass/fail, and deliver comparison reports through your agent. [Get started](https://www.stet.sh/private)

## About this data

- 24 tasks, each a merged change from the private `Stet-AI/stet` repository replayed from the repository state immediately before the change.
- Data from **November 19, 2026**.
- Both setups came from the same leaderboard run of 18 setups: `bmk_d4ba310fdf50db4ccb2092e3`.

[See all 18 setups on the leaderboard](https://www.stet.sh/leaderboard) · [How Stet measures](https://www.stet.sh/methodology)

## More comparisons

- [Pareto 26.9 medium vs Grok 4.7 high](https://www.stet.sh/leaderboard/compare/grok-4-7-high-vs-pareto-26-9-medium)
- [Pareto 26.9 medium vs DeepSeek V4.1 Flash max](https://www.stet.sh/leaderboard/compare/deepseek-v4-1-flash-max-vs-pareto-26-9-medium)
- [Pareto 26.9 medium vs GPT 6 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-pareto-26-9-medium)
- [Pareto 26.9 medium vs DeepSeek V4.1 Flash (10/17) max](https://www.stet.sh/leaderboard/compare/deepseek-v4-1-flash-10-17-max-vs-pareto-26-9-medium)
- [Pareto 26.9 medium vs GPT 6.1 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-pareto-26-9-medium)
- [SWE-2 Max high vs Opus 5 medium](https://www.stet.sh/leaderboard/compare/opus-5-medium-vs-swe-2-max-high)
- [SWE-2 Max high vs Muse Wonder Full low](https://www.stet.sh/leaderboard/compare/muse-wonder-full-low-vs-swe-2-max-high)
- [SWE-2 Max high vs GPT 6.1 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-swe-2-max-high)
- [SWE-2 Max high vs GPT 6 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-swe-2-max-high)
- [SWE-2 Max high vs Grok 4.7 high](https://www.stet.sh/leaderboard/compare/grok-4-7-high-vs-swe-2-max-high)
- [All 153 comparisons](https://www.stet.sh/leaderboard/compare)

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fpareto-26-9-medium-vs-swe-2-max-high&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fpareto-26-9-medium-vs-swe-2-max-high). Opening a form does not execute a tool.