# GPT 6 Sol medium vs Pareto 26.9 medium

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-pareto-26-9-medium
Source observed: 2026-11-19T11:08:22.654Z
Last checked: 2026-11-19T11:08:22.654+00:00
Indexed: 2026-11-19T11:18:24.393+00:00

---

# GPT 6 Sol medium vs Pareto 26.9 medium

We compare GPT 6 Sol medium and Pareto 26.9 medium as complete coding-agent setups on 24 merged changes from our private `Stet-AI/stet` repository. GPT 6 Sol medium used **codex** with **medium** reasoning; Pareto 26.9 medium used **claude-code** with **medium** reasoning.

## Summary

- **Quality:** Even; each setup won 3 head-to-head tasks and 18 were tied.
- **Speed:** GPT 6 Sol medium was **2.5× faster**: 15.4 versus 39.2 minutes per task.
- **Cost:** GPT 6 Sol medium was **2.0× cheaper**: $3.01 versus $5.98 per task.

## Test scope and scores

We replayed 24 merged changes from private `Stet-AI/stet`, beginning each run from the repository state before the change. Repository tests and judges compared patches with the merged changes. Data: **November 19, 2026**. Both setups came from leaderboard run `bmk_d4ba310fdf50db4ccb2092e3`, containing 18 setups.

| Setup | Harness | Reasoning | Rank | Stet score | 95% range |
|---|---|---:|---:|---:|---:|
| GPT 6 Sol medium | codex | medium | 5 of 18 | 53.4 | 47.5–59.3 |
| Pareto 26.9 medium | claude-code | medium | 7 of 18 | 52.3 | 44.9–59.5 |

The score ranges overlap. Stet score represents a setup’s average chance of beating another setup on a task; 50 is even with the field.

## Quality

GPT 6 Sol medium won 3 matchups, Pareto won 3, and 18 were tied. GPT’s win share was 52%, with a 95% range of 42–61%. Ties are within judge noise.

| Metric | GPT 6 Sol medium | Pareto 26.9 medium |
|---|---:|---:|
| Code review pass rate | 35% (23 tasks) | 35% (23 tasks) |
| Checklist score | 0.86 (24 tasks) | 0.80 (24 tasks) |
| Equivalence rate | 43% (23 tasks) | 39% (23 tasks) |

These additional measures are not part of the headline Stet score and have no confidence interval.

## Task by task

GPT solved 2 tasks Pareto did not; Pareto solved 1 GPT did not. Both solved 10 and neither solved 11. Among the 10 both solved, judges preferred Pareto’s patch on 2, GPT’s on 1, and called 7 even.

### Only GPT 6 Sol medium solved

- `stet-commit-5d766dde` — 218 changed lines in 2 files; judges preferred GPT’s patch.
- `stet-commit-342d0ac0` — 640 changed lines in 8 files; judges preferred GPT’s patch.

### Only Pareto 26.9 medium solved

- `stet-2450ca2d` — 2 changed lines in 1 file; judges preferred Pareto’s patch.

### Both solved

- `stet-43493c52` — 4 changed lines in 1 file.
- `stet-3169aefa` — 6 changed lines in 1 file.
- `stet-89dfbc27` — 9 changed lines in 1 file.
- `stet-232b50ba` — 10 changed lines in 1 file.
- `stet-5276aa3f` — 23 changed lines in 2 files.
- `stet-b28f4fe7` — 29 changed lines in 1 file.
- `stet-66762b4c` — 84 changed lines in 1 file.
- `stet-commit-f0caded9` — 113 changed lines in 4 files; judges preferred Pareto’s patch.
- `stet-commit-bbbbae09` — 941 changed lines in 7 files; judges preferred Pareto’s patch.
- `stet-commit-f8ae1177` — 1,312 changed lines in 9 files; judges preferred GPT’s patch.

### Neither solved

- `stet-15439c21` — 5 changed lines in 2 files.
- `stet-6f84e978` — 23 changed lines in 1 file.
- `stet-commit-e928166f` — 149 changed lines in 2 files.
- `stet-commit-30506f8e` — 476 changed lines in 17 files.
- `stet-commit-5e075641` — 481 changed lines in 3 files.
- `stet-commit-60a0e6c2` — 552 changed lines in 11 files.
- `stet-commit-a8716b88` — 564 changed lines in 10 files.
- `stet-commit-87814583` — 763 changed lines in 8 files.
- `stet-commit-e41c2e2d` — 977 changed lines in 5 files.
- `stet-commit-d2ab0c25` — 1,236 changed lines in 16 files.
- `stet-commit-2826765e` — 1,420 changed lines in 19 files.

A task is solved when tests pass or a judge finds the fix equivalent to the merged change. Stet score also weighs code quality.

## Cost and time

GPT cost less on 19 of 24 tasks; Pareto cost less on 5. Mean and median cost were $3.01 and $1.74 for GPT, versus $5.98 and $3.66 for Pareto. GPT finished faster on 20 of 24 tasks; Pareto was faster on 4. Mean and median time were 15.4 and 12.7 minutes for GPT, versus 39.2 and 38.2 minutes for Pareto.

## Where each setup is the better choice

### Small fixes

Ten tasks covered 2–84 changed lines in 1–2 files. Pareto solved 8/10 versus GPT’s 7/10; GPT cost 4.0× less and was 4.7× faster.

| | GPT | Pareto |
|---|---:|---:|
| Solved | 7/10 | 8/10 |
| Cost / task | $0.61 | $2.42 |
| Agent time / task | 6.2 min | 29.0 min |

### Large, cross-cutting changes

Fourteen tasks covered 113–1,420 changed lines in up to 19 files. GPT solved 5/14 versus Pareto’s 3/14, cost 1.8× less, and was 2.1× faster.

| | GPT | Pareto |
|---|---:|---:|
| Solved | 5/14 | 3/14 |
| Cost / task | $4.72 | $8.53 |
| Agent time / task | 22.0 min | 46.5 min |

## Side by side

| Metric | GPT 6 Sol medium | Pareto 26.9 medium |
|---|---|---|
| Harness | codex | claude-code |
| Reasoning effort | medium | medium |
| Rank | 5 of 18 | 7 of 18 |
| Stet score | 53.4; 95% range 47.5–59.3 | 52.3; 95% range 44.9–59.5 |
| Solved | 12/24: 9 passed, 3 judged equivalent | 11/24: 10 passed, 1 judged equivalent |
| Cost / task | $3.01 | $5.98 |
| Agent time / task | 15.4 min | 39.2 min |

## Questions

**Which is better for coding?** We found no clear quality difference: each won 3 and 18 tied. GPT was 2.0× cheaper and 2.5× faster on the 24 merged changes.

**How were they tested?** Both agents worked on the same 24 merged changes from the private repository, from the real state before each change. Repository tests and judges graded each patch against the merged change; no public benchmark scores were used.

## See which setup wins on your repo

We replay your merged changes, score quality above pass/fail, and deliver comparison reports through your agent. [Get started](https://www.stet.sh/private)

## Links

- [See all 18 setups](https://www.stet.sh/leaderboard)
- [How Stet measures](https://www.stet.sh/methodology)
- [GPT 6 Sol medium vs GPT 6.1 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-gpt-6-sol-medium)
- [GPT 6 Sol medium vs Grok 4.7 high](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-grok-4-7-high)
- [GPT 6 Sol medium vs Muse Wonder Full low](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-muse-wonder-full-low)
- [GPT 6 Sol medium vs Opus 5 medium](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-opus-5-medium)
- [GPT 6 Sol medium vs DeepSeek V4.1 Flash max](https://www.stet.sh/leaderboard/compare/deepseek-v4-1-flash-max-vs-gpt-6-sol-medium)
- [Pareto 26.9 medium vs Grok 4.7 high](https://www.stet.sh/leaderboard/compare/grok-4-7-high-vs-pareto-26-9-medium)
- [Pareto 26.9 medium vs DeepSeek V4.1 Flash max](https://www.stet.sh/leaderboard/compare/deepseek-v4-1-flash-max-vs-pareto-26-9-medium)
- [Pareto 26.9 medium vs DeepSeek V4.1 Flash (10/17) max](https://www.stet.sh/leaderboard/compare/deepseek-v4-1-flash-10-17-max-vs-pareto-26-9-medium)
- [Pareto 26.9 medium vs GPT 6.1 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-pareto-26-9-medium)
- [Pareto 26.9 medium vs Haiku 5.5 max](https://www.stet.sh/leaderboard/compare/haiku-5-5-max-vs-pareto-26-9-medium)
- [All 153 comparisons](https://www.stet.sh/leaderboard/compare)

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-6-sol-medium-vs-pareto-26-9-medium&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-6-sol-medium-vs-pareto-26-9-medium). Opening a form does not execute a tool.