# Opus 4.8 medium vs Pareto 26.9 medium

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/leaderboard/compare/opus-4-8-medium-vs-pareto-26-9-medium
Source observed: 2026-11-19T11:08:36.627Z
Last checked: 2026-11-19T11:08:36.627+00:00
Indexed: 2026-11-19T11:15:07.565+00:00

---

# Opus 4.8 medium vs Pareto 26.9 medium

We compare two coding-agent setups on real repository work. **Opus 4.8 medium leads on speed; Pareto 26.9 medium leads on cost; we found no clear quality difference.**

## Headline results

| Dimension | Result |
| --- | --- |
| Quality | Even: no clear difference. Opus won 4 head-to-head tasks, Pareto won 6, 12 were tied, and 2 were undecided. |
| Speed | Opus 4.8 medium was 1.7× faster: 23.3 minutes versus 39.2 minutes per task. |
| Cost | Pareto 26.9 medium was 1.3× cheaper: $5.98 versus $7.68 per task. |

## What we compared

- **1 real repository:** private `Stet-AI/stet`; these are not public benchmark scores.
- **24 merged changes:** each setup ran end to end from the repository state before the change.
- **Grading:** the repository’s tests and judges comparing each patch with the merged change.
- **Measures:** quality, cost, and time on a smaller sample than public benchmarks, but on real agent work in a real codebase.

Data covers 24 tasks from November 19, 2026. Both setups used the `claude-code` harness with medium reasoning.

## Stet score

| Setup | Rank | Stet score | 95% range |
| --- | ---: | ---: | --- |
| Opus 4.8 medium | 13 of 18 | 46.8 | 38.0–55.7 |
| Pareto 26.9 medium | 7 of 18 | 52.3 | 44.9–59.5 |

The 95% score ranges overlap. Stet score is a setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field.

## Quality

Pareto’s head-to-head win share was 54%, with a 95% range of 40–67%. Ties are within judge noise.

### Other quality measures

| Metric | Opus 4.8 medium | Pareto 26.9 medium |
| --- | --- | --- |
| Code review pass rate | 33% — 24 tasks | 35% — 23 tasks |
| Checklist score | 0.78 — 24 tasks | 0.80 — 24 tasks |
| Equivalence rate | 46% — 24 tasks | 39% — 23 tasks |

These measures are not part of the headline Stet score and have no confidence interval; a small gap may be noise. The higher recorded value is not a proven lead.

## Task-by-task outcome

Opus solved 3 tasks that Pareto did not. Pareto solved 1 that Opus did not. Both solved 10; neither solved 8. On the 10 tasks both solved, judges preferred Pareto’s patch on 5, Opus’s on 1, and called 4 even.

### Only Opus 4.8 medium solved

- 5 changed lines in 2 files — `stet-15439c21`; judges preferred Opus’s patch.
- 23 changed lines in 1 file — `stet-6f84e978`; judges preferred Opus’s patch.
- 218 changed lines in 2 files — `stet-commit-5d766dde`; judges preferred Opus’s patch.

### Only Pareto 26.9 medium solved

- 2 changed lines in 1 file — `stet-2450ca2d`; judges preferred Pareto’s patch.

### Both solved

- 4 changed lines in 1 file — `stet-43493c52`; judges preferred Pareto’s patch.
- 6 changed lines in 1 file — `stet-3169aefa`.
- 9 changed lines in 1 file — `stet-89dfbc27`; judges preferred Pareto’s patch.
- 10 changed lines in 1 file — `stet-232b50ba`; judges preferred Pareto’s patch.
- 23 changed lines in 2 files — `stet-5276aa3f`.
- 29 changed lines in 1 file — `stet-b28f4fe7`; judges preferred Pareto’s patch.
- 84 changed lines in 1 file — `stet-66762b4c`; judges preferred Opus’s patch.
- 113 changed lines in 4 files — `stet-commit-f0caded9`.
- 941 changed lines in 7 files — `stet-commit-bbbbae09`; judges preferred Pareto’s patch.
- 1,312 changed lines in 9 files — `stet-commit-f8ae1177`.

### Neither solved

- 149 changed lines in 2 files — `stet-commit-e928166f`
- 476 changed lines in 17 files — `stet-commit-30506f8e`
- 481 changed lines in 3 files — `stet-commit-5e075641`
- 564 changed lines in 10 files — `stet-commit-a8716b88`
- 640 changed lines in 8 files — `stet-commit-342d0ac0`
- 763 changed lines in 8 files — `stet-commit-87814583`
- 977 changed lines in 5 files — `stet-commit-e41c2e2d`
- 1,420 changed lines in 19 files — `stet-commit-2826765e`

A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change. Stet score also weighs code quality, so it can order setups differently. Two of 24 tasks were left out because one setup had no scored result on them.

## Cost and time

On the 22 compared tasks:

- **Cost:** Opus mean $7.46, median $3.18; Pareto mean $6.22, median $3.66. Opus cost less on 13 tasks; Pareto on 9.
- **Agent time:** Opus mean 23.8 minutes, median 17.3; Pareto mean 41.4 minutes, median 41.5. Opus finished faster on 17 tasks; Pareto on 5.

## Where each setup may fit

These groups contain fewer tasks than the overall standings and are leads to test on your repository.

### Small fixes

10 tasks, with 2–84 changed lines in 1–2 files. Opus solved 9/10 versus Pareto’s 8/10, at 2.0× lower cost per task, and was 2.5× faster.

| Metric | Opus 4.8 medium | Pareto 26.9 medium |
| --- | ---: | ---: |
| Solved | 9/10 | 8/10 |
| Cost / task | $1.22 | $2.42 |
| Agent time / task | 11.6 min | 29.0 min |

### Large, cross-cutting changes

14 tasks, with 113–1,420 changed lines in up to 19 files. Opus solved 4/12 versus Pareto’s 3/12; Pareto cost 1.3× less per task. Opus was 1.5× faster.

| Metric | Opus 4.8 medium | Pareto 26.9 medium |
| --- | ---: | ---: |
| Solved | 4/12 | 3/12 |
| Cost / task | $12.66 | $9.38 |
| Agent time / task | 34.0 min | 51.8 min |

## Side by side

| Metric | Opus 4.8 medium | Pareto 26.9 medium |
| --- | --- | --- |
| Harness | `claude-code` | `claude-code` |
| Reasoning effort | medium | medium |
| Rank | 13 of 18 | 7 of 18 |
| Stet score | 46.8; 95% range 38.0–55.7 | 52.3; 95% range 44.9–59.5 |
| Solved | 13/24: 11 passed, 2 judged equivalent | 11/24: 10 passed, 1 judged equivalent |
| Cost / task | $7.68 | $5.98 |
| Agent time / task | 23.3 min | 39.2 min |

## Further reading

- [Cost per successful task ranked the coding agent that lost every head to head 1st](https://www.stet.sh/blog/cost-per-successful-task)
- [GLM 5.2 vs Composer 2.5 on 50 real Go and Rust PRs: last on quality, and not the cheapest](https://www.stet.sh/blog/glm-5-2-passes-tests-fails-review)
- [Composer 2.5 vs Opus 4.8 vs GPT-5.5 vs Opus 4.7 - 50 Real PRs in Go and Rust](https://www.stet.sh/blog/opus-48-vs-gpt-55-vs-opus-47-vs-composer-25)

## Questions

**Which is better for coding?** We found no clear quality difference. Pareto won 6 head-to-head tasks, Opus won 4, 12 were tied, and 2 were undecided. Pareto leads on cost by 1.3×; Opus leads on agent time by 1.7×.

**How did we test them?** Both coding agents worked the same 24 merged changes in the private `Stet-AI/stet` repository, each from the real repository state before the change. The repository tests and judges graded every patch against the merged change; we used no public benchmark scores.

## Test your repository

We replay your merged changes, score quality above pass/fail, and deliver comparison reports through your agent. [Get started](https://www.stet.sh/private)

## About this data

- 24 merged changes from private `Stet-AI/stet`, replayed from the repository state just before each change.
- Data from November 19, 2026.
- Both setups came from leaderboard run `bmk_d4ba310fdf50db4ccb2092e3`, containing 18 setups.

Links: [See all 18 setups](https://www.stet.sh/leaderboard) · [How Stet measures](https://www.stet.sh/methodology)

### More comparisons

- [Opus 4.8 medium vs GLM 5.3 Flash high](https://www.stet.sh/leaderboard/compare/glm-5-3-flash-high-vs-opus-4-8-medium)
- [Opus 4.8 medium vs GPT 5.6 Luna max](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-max-vs-opus-4-8-medium)
- [Opus 4.8 medium vs Muse Spark 1.3 high](https://www.stet.sh/leaderboard/compare/muse-spark-1-3-high-vs-opus-4-8-medium)
- [Opus 4.8 medium vs GPT 6 Luna max](https://www.stet.sh/leaderboard/compare/gpt-6-luna-max-vs-opus-4-8-medium)
- [Opus 4.8 medium vs Haiku 5.5 max](https://www.stet.sh/leaderboard/compare/haiku-5-5-max-vs-opus-4-8-medium)
- [Pareto 26.9 medium vs Grok 4.7 high](https://www.stet.sh/leaderboard/compare/grok-4-7-high-vs-pareto-26-9-medium)
- [Pareto 26.9 medium vs DeepSeek V4.1 Flash max](https://www.stet.sh/leaderboard/compare/deepseek-v4-1-flash-max-vs-pareto-26-9-medium)
- [Pareto 26.9 medium vs GPT 6 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-pareto-26-9-medium)
- [Pareto 26.9 medium vs DeepSeek V4.1 Flash (10/17) max](https://www.stet.sh/leaderboard/compare/deepseek-v4-1-flash-10-17-max-vs-pareto-26-9-medium)
- [Pareto 26.9 medium vs GPT 6.1 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-pareto-26-9-medium)
- [All 153 comparisons](https://www.stet.sh/leaderboard/compare)

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fopus-4-8-medium-vs-pareto-26-9-medium&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fopus-4-8-medium-vs-pareto-26-9-medium). Opening a form does not execute a tool.