# GPT 6 Sol medium vs Haiku 5.5 max: real-repo coding test

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-haiku-5-5-max
Source observed: 2026-11-19T11:08:40.465Z
Last checked: 2026-11-19T11:08:40.465+00:00
Indexed: 2026-11-19T11:18:18.136+00:00

---

# GPT 6 Sol medium vs Haiku 5.5 max

We compare two coding-agent setups on real work from our private `Stet-AI/stet` repository. GPT 6 Sol medium used **codex** with **medium reasoning**; Haiku 5.5 max used **claude-code** with **max reasoning**. This compares complete setups, not models alone.

## At a glance

- **Quality:** Even; no clear difference. Each setup won 5 head-to-head tasks, 13 tied, and 1 was undecided.
- **Speed:** GPT 6 Sol medium was **3.8× faster**: 15.4 minutes versus 58.4 minutes per task.
- **Cost:** GPT 6 Sol medium was **1.3× cheaper**: $3.01 versus $3.83 per task.

## What we compared

- **1 real repository:** private `Stet-AI/stet`; not public benchmark scores.
- **24 merged changes:** each setup ran end to end from the repository state before the change.
- **Grading:** the repository’s tests and judges compared each patch with the merged change.
- **Measures:** quality, cost, and time from real agent work on a real codebase.

[GPT 6 Sol vs Haiku 5.5: all effort pairings](https://www.stet.sh/leaderboard/compare/gpt-6-sol-vs-haiku-5-5)

Data: **November 19, 2026** · 24 tasks.

## Stet score

| Setup | Harness and reasoning | Rank | Stet score |
|---|---|---:|---:|
| GPT 6 Sol medium | codex · medium | 5 of 18 | 53.4; 95% range 47.5–59.3 |
| Haiku 5.5 max | claude-code · max | 10 of 18 | 48.7; 95% range 38.3–58.7 |

The 95% score ranges overlap. Stet score is a setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field.

## Quality

GPT 6 Sol medium won 5 head-to-head tasks, Haiku 5.5 max won 5, 13 were tied, and 1 was undecided. GPT 6 Sol medium’s win share was 52%, with a 95% range of 39–65%. Ties are within judge noise.

### Other quality measures

| Metric | GPT 6 Sol medium | Haiku 5.5 max |
|---|---:|---:|
| Code review pass rate | 35% · 23 tasks | 38% · 24 tasks |
| Checklist score | 0.86 · 24 tasks | 0.81 · 24 tasks |
| Equivalence rate | 43% · 23 tasks | 42% · 24 tasks |

These measures are not part of the headline Stet score and have no confidence interval. A small gap may be noise; a higher recorded value is not a proven lead.

## Task by task

GPT 6 Sol medium solved 5 tasks that Haiku 5.5 max did not. Haiku 5.5 max solved 3 that GPT 6 Sol medium did not. Both solved 7; neither solved 8. On the 7 tasks both solved, judges preferred Haiku 5.5 max’s patch on 1 and called 6 even.

### Only GPT 6 Sol medium solved

- 4 changed lines in 1 file — `stet-43493c52`; judges preferred GPT 6 Sol medium’s patch.
- 10 changed lines in 1 file — `stet-232b50ba`; judges preferred GPT 6 Sol medium’s patch.
- 84 changed lines in 1 file — `stet-66762b4c`; judges preferred GPT 6 Sol medium’s patch.
- 113 changed lines in 4 files — `stet-commit-f0caded9`; judges preferred GPT 6 Sol medium’s patch.
- 1,312 changed lines in 9 files — `stet-commit-f8ae1177`; judges preferred GPT 6 Sol medium’s patch.

### Only Haiku 5.5 max solved

- 2 changed lines in 1 file — `stet-2450ca2d`; judges preferred Haiku 5.5 max’s patch.
- 5 changed lines in 2 files — `stet-15439c21`; judges preferred Haiku 5.5 max’s patch.
- 23 changed lines in 1 file — `stet-6f84e978`; judges preferred Haiku 5.5 max’s patch.

### Both solved

- 6 changed lines in 1 file — `stet-3169aefa`.
- 9 changed lines in 1 file — `stet-89dfbc27`.
- 23 changed lines in 2 files — `stet-5276aa3f`.
- 29 changed lines in 1 file — `stet-b28f4fe7`.
- 218 changed lines in 2 files — `stet-commit-5d766dde`.
- 640 changed lines in 8 files — `stet-commit-342d0ac0`; judges preferred Haiku 5.5 max’s patch.
- 941 changed lines in 7 files — `stet-commit-bbbbae09`.

### Neither solved

- 149 changed lines in 2 files — `stet-commit-e928166f`; judges preferred Haiku 5.5 max’s patch.
- 476 changed lines in 17 files — `stet-commit-30506f8e`.
- 481 changed lines in 3 files — `stet-commit-5e075641`.
- 564 changed lines in 10 files — `stet-commit-a8716b88`.
- 763 changed lines in 8 files — `stet-commit-87814583`.
- 977 changed lines in 5 files — `stet-commit-e41c2e2d`.
- 1,236 changed lines in 16 files — `stet-commit-d2ab0c25`.
- 1,420 changed lines in 19 files — `stet-commit-2826765e`.

A task is solved when its tests pass or a judge finds the fix equivalent to the merged change. Stet score also weighs code quality, so it can order setups differently. One of the 24 tasks is excluded because one setup has no scored result; the comparison table therefore contains 23 tasks.

## Cost and time on each task

Each task is shown on a log scale; filled markers indicate solved tasks and hollow markers indicate unsolved tasks. The line marks the median.

- **Cost:** GPT 6 Sol medium mean $2.92, median $1.38; Haiku 5.5 max mean $3.77, median $3.05. GPT 6 Sol medium cost less on 16 of 23 tasks; Haiku 5.5 max cost less on 7.
- **Agent time:** GPT 6 Sol medium mean 15.3 minutes, median 11.9; Haiku 5.5 max mean 59.7 minutes, median 47.8. GPT 6 Sol medium finished faster on 22 of 23 tasks; Haiku 5.5 max was faster on 1.

## Where each setup is the better choice

These groups contain fewer tasks than the overall standings, so use them as leads to test on your repository.

### Small fixes

10 tasks, with 2–84 changed lines in 1–2 files. Both setups solved 7/10; GPT 6 Sol medium cost 1.8× less per task and was 5.8× faster.

| Metric | GPT 6 Sol medium | Haiku 5.5 max |
|---|---:|---:|
| Solved | 7/10 | 7/10 |
| Cost / task | $0.61 | $1.11 |
| Agent time / task | 6.2 min | 36.4 min |

### Large, cross-cutting changes

14 tasks, with 113–1,420 changed lines in up to 19 files. GPT 6 Sol medium solved 5/13 versus Haiku 5.5 max’s 3/13, cost 1.2× less per task, and was 3.5× faster.

| Metric | GPT 6 Sol medium | Haiku 5.5 max |
|---|---:|---:|
| Solved | 5/13 | 3/13 |
| Cost / task | $4.69 | $5.82 |
| Agent time / task | 22.4 min | 77.6 min |

## Side by side

| Metric | GPT 6 Sol medium | Haiku 5.5 max |
|---|---|---|
| Harness | codex | claude-code |
| Reasoning effort | medium | max |
| Rank | 5 of 18 | 10 of 18 |
| Stet score | 53.4; 95% range 47.5–59.3 | 48.7; 95% range 38.3–58.7 |
| Solved | 12/24; 9 passed, 3 judged equivalent | 10/24; 7 passed, 3 judged equivalent |
| Cost / task | $3.01 | $3.83 |
| Agent time / task | 15.4 min | 58.4 min |

## Questions

**Which is better for coding?** There is no clear quality difference: GPT 6 Sol medium won 5, Haiku 5.5 max won 5, 13 tied, and 1 was undecided. GPT 6 Sol medium was 1.3× cheaper and 3.8× faster, measured on 24 merged changes from `Stet-AI/stet`.

**How did we test them?** Both coding agents worked on the same 24 merged changes in the private repository, starting from the real repository state before each change. The repository’s tests and judges graded patches against the merged change; no public benchmark scores were used.

## Run this comparison on your repository

We replay your merged changes, score quality above pass/fail, and deliver comparison reports through your agent. [Get started](https://www.stet.sh/private)

## About this data

- 24 tasks, each a merged change from the private `Stet-AI/stet` repository, replayed from the state just before the change.
- Data from November 19, 2026.
- Both setups came from leaderboard run `bmk_d4ba310fdf50db4ccb2092e3`, containing 18 setups.

[See all 18 setups on the leaderboard](https://www.stet.sh/leaderboard) · [How Stet measures](https://www.stet.sh/methodology)

## More comparisons

- [GPT 6 Sol medium vs GPT 6.1 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-gpt-6-sol-medium)
- [GPT 6 Sol medium vs Grok 4.7 high](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-grok-4-7-high)
- [GPT 6 Sol medium vs Muse Wonder Full low](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-muse-wonder-full-low)
- [GPT 6 Sol medium vs Pareto 26.9 medium](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-pareto-26-9-medium)
- [GPT 6 Sol medium vs Opus 5 medium](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-opus-5-medium)
- [Haiku 5.5 max vs DeepSeek V4.1 Flash (10/17) max](https://www.stet.sh/leaderboard/compare/deepseek-v4-1-flash-10-17-max-vs-haiku-5-5-max)
- [Haiku 5.5 max vs Muse Spark 1.3 high](https://www.stet.sh/leaderboard/compare/haiku-5-5-max-vs-muse-spark-1-3-high)
- [Haiku 5.5 max vs DeepSeek V4.1 Flash max](https://www.stet.sh/leaderboard/compare/deepseek-v4-1-flash-max-vs-haiku-5-5-max)
- [Haiku 5.5 max vs GLM 5.3 Flash high](https://www.stet.sh/leaderboard/compare/glm-5-3-flash-high-vs-haiku-5-5-max)
- [Haiku 5.5 max vs Pareto 26.9 medium](https://www.stet.sh/leaderboard/compare/haiku-5-5-max-vs-pareto-26-9-medium)
- [All 153 comparisons](https://www.stet.sh/leaderboard/compare)

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-6-sol-medium-vs-haiku-5-5-max&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-6-sol-medium-vs-haiku-5-5-max). Opening a form does not execute a tool.