# GPT 6 Sol medium vs Haiku 5.5 low

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-haiku-5-5-low
Source observed: 2026-11-19T11:08:38.322Z
Last checked: 2026-11-19T11:08:38.322+00:00
Indexed: 2026-11-19T11:17:50.588+00:00

---

# GPT 6 Sol medium vs Haiku 5.5 low

Haiku 5.5 low leads on speed and cost; quality is not clear from this comparison. The setups use different harnesses: GPT 6 Sol medium ran `codex` with medium reasoning, while Haiku 5.5 low ran `claude-code` with low reasoning. We compare setups, not models alone.

## Headline results

| Dimension | Result |
|---|---|
| Quality | GPT 6 Sol medium won 5, Haiku 5.5 low won 2, and 17 were tied. |
| Speed | Haiku 5.5 low was **1.6× faster**: 9.9 vs 15.4 minutes per task. |
| Cost | Haiku 5.5 low was **12× cheaper**: $0.25 vs $3.01 per task. |

## What we compared

- **1 real repository:** private `Stet-AI/stet`; not public benchmark scores.
- **24 merged changes:** each setup ran end to end from the repository state before the change.
- **Grading:** the repository’s own tests and judges comparing each patch with the merged change.
- **Measures:** quality, cost, and time on real agent work; the sample is smaller than public benchmarks.

For all effort pairings, see [GPT 6 Sol vs Haiku 5.5](https://www.stet.sh/leaderboard/compare/gpt-6-sol-vs-haiku-5-5). Data is from **November 19, 2026**.

## Stet score and quality

| Setup | Harness / reasoning | Rank | Stet score | 95% range |
|---|---|---:|---:|---:|
| GPT 6 Sol medium | `codex` / medium | 5 of 18 | 53.4 | 47.5–59.3 |
| Haiku 5.5 low | `claude-code` / low | 16 of 18 | 44.9 | 39.1–50.3 |

The 95% score ranges overlap. GPT 6 Sol medium’s win share is 56%, with a 95% range of 45–67%; ties are within judge noise.

### Other quality measures

| Metric | GPT 6 Sol medium | Haiku 5.5 low |
|---|---:|---:|
| Code review pass rate | 35% (23 tasks) | 33% (24 tasks) |
| Checklist score | 0.86 (24 tasks) | 0.77 (24 tasks) |
| Equivalence rate | 43% (23 tasks) | 38% (24 tasks) |

These measures are not part of the headline Stet score and have no confidence interval; a small gap may be noise. The higher recorded value is not a proven lead.

## Task by task

GPT 6 Sol medium solved 2 tasks that Haiku did not. Haiku solved 1 that GPT 6 Sol medium did not. Both solved 10, and neither solved 11. On the 10 tasks both solved, judges preferred GPT 6 Sol medium’s patch on 3, Haiku’s on 1, and called 6 even.

### Only GPT 6 Sol medium solved

- `stet-commit-342d0ac0` — 640 changed lines in 8 files; judges preferred GPT 6 Sol medium’s patch.
- `stet-commit-f8ae1177` — 1,312 changed lines in 9 files; judges preferred GPT 6 Sol medium’s patch.

### Only Haiku 5.5 low solved

- `stet-15439c21` — 5 changed lines in 2 files; judges preferred Haiku’s patch.

### Both solved

- `stet-43493c52` — 4 lines, 1 file; GPT 6 Sol medium preferred.
- `stet-3169aefa` — 6 lines, 1 file.
- `stet-89dfbc27` — 9 lines, 1 file.
- `stet-232b50ba` — 10 lines, 1 file.
- `stet-5276aa3f` — 23 lines, 2 files.
- `stet-b28f4fe7` — 29 lines, 1 file.
- `stet-66762b4c` — 84 lines, 1 file; GPT 6 Sol medium preferred.
- `stet-commit-f0caded9` — 113 lines, 4 files; Haiku preferred.
- `stet-commit-5d766dde` — 218 lines, 2 files.
- `stet-commit-bbbbae09` — 941 lines, 7 files; GPT 6 Sol medium preferred.

### Neither solved

- `stet-2450ca2d` — 2 lines, 1 file.
- `stet-6f84e978` — 23 lines, 1 file.
- `stet-commit-e928166f` — 149 lines, 2 files.
- `stet-commit-30506f8e` — 476 lines, 17 files.
- `stet-commit-5e075641` — 481 lines, 3 files.
- `stet-commit-60a0e6c2` — 552 lines, 11 files.
- `stet-commit-a8716b88` — 564 lines, 10 files.
- `stet-commit-87814583` — 763 lines, 8 files.
- `stet-commit-e41c2e2d` — 977 lines, 5 files.
- `stet-commit-d2ab0c25` — 1,236 lines, 16 files.
- `stet-commit-2826765e` — 1,420 lines, 19 files.

A task counts as solved when tests pass or a judge finds the fix equivalent to the merged change; Stet score also weighs code quality.

## Cost and time on each task

The comparison uses a log scale; filled points indicate solved tasks, hollow points indicate unsolved tasks, and the line marks the median.

- **GPT 6 Sol medium:** mean $3.01, median $1.74.
- **Haiku 5.5 low:** mean $0.25, median $0.04.
- Haiku cost less on all 24 tasks.
- **GPT 6 Sol medium:** mean 15.4 minutes, median 12.7 minutes.
- **Haiku 5.5 low:** mean 9.9 minutes, median 4.7 minutes.
- Haiku finished faster on 20 of 24 tasks; GPT 6 Sol medium finished faster on 4.

## Where each setup is the better choice

### Small fixes

10 tasks, 2–84 changed lines in 1–2 files. Haiku solved 8/10 versus GPT 6 Sol medium’s 7/10, at 42× lower cost and 2.3× faster.

| Metric | GPT 6 Sol medium | Haiku 5.5 low |
|---|---:|---:|
| Solved | 7/10 | 8/10 |
| Cost / task | $0.61 | $0.01 |
| Agent time / task | 6.2 min | 2.7 min |

### Large, cross-cutting changes

14 tasks, 113–1,420 changed lines in up to 19 files. GPT 6 Sol medium solved 5/14 versus Haiku’s 3/14; Haiku cost 11× less and was 1.5× faster.

| Metric | GPT 6 Sol medium | Haiku 5.5 low |
|---|---:|---:|
| Solved | 5/14 | 3/14 |
| Cost / task | $4.72 | $0.42 |
| Agent time / task | 22.0 min | 15.0 min |

## Side by side

| Metric | GPT 6 Sol medium | Haiku 5.5 low |
|---|---|---|
| Harness | `codex` | `claude-code` |
| Reasoning effort | medium | low |
| Rank | 5 of 18 | 16 of 18 |
| Stet score | 53.4, 95% range 47.5–59.3 | 44.9, 95% range 39.1–50.3 |
| Solved | 12/24: 9 passed, 3 judged equivalent | 11/24: 7 passed, 4 judged equivalent |
| Cost / task | $3.01 | $0.25 |
| Agent time / task | 15.4 min | 9.9 min |

## Research links

- [Cost per successful task ranked the coding agent that lost every head to head 1st](https://www.stet.sh/blog/cost-per-successful-task)
- [A benchmark mined from my own repo agreed with FrontierCode](https://www.stet.sh/blog/benchmark-agreed-with-frontier-code)
- [GPT 6.1 Sol vs GPT 6 Sol on 24 tasks](https://www.stet.sh/blog/gpt-6-1-sol-vs-gpt-6-sol)

## Questions

**Which is better for coding?** The comparison shows no clear quality difference: GPT 6 Sol medium won 5, Haiku 5.5 low won 2, and 17 tied. Haiku was 12× cheaper and 1.6× faster.

**How did we test them?** Both agents worked on the same 24 merged changes from private `Stet-AI/stet`, each from the repository state before the change. Tests and judges graded every patch against the merged change; no public benchmark scores were used. A Stet score is a setup’s average chance of beating another setup on a task from 0 to 100; 50 is even with the field.

## Run this comparison on your repository

Stet replays your merged changes, scores quality above pass/fail, and delivers comparison reports through your agent. [Get started](https://www.stet.sh/private)

## About this data

- 24 merged changes from private `Stet-AI/stet`, replayed from the repository state just before each change.
- Data from November 19, 2026.
- Same leaderboard run of 18 setups: `bmk_d4ba310fdf50db4ccb2092e3`.

Links: [See all 18 setups](https://www.stet.sh/leaderboard) · [How Stet measures](https://www.stet.sh/methodology)

## More comparisons

- [GPT 6 Sol medium vs GPT 6.1 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-gpt-6-sol-medium)
- [GPT 6 Sol medium vs Grok 4.7 high](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-grok-4-7-high)
- [GPT 6 Sol medium vs Muse Wonder Full low](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-muse-wonder-full-low)
- [GPT 6 Sol medium vs Pareto 26.9 medium](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-pareto-26-9-medium)
- [GPT 6 Sol medium vs Opus 5 medium](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-opus-5-medium)
- [Haiku 5.5 low vs GPT 6 Luna max](https://www.stet.sh/leaderboard/compare/gpt-6-luna-max-vs-haiku-5-5-low)
- [Haiku 5.5 low vs GPT 5.6 Luna low](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-haiku-5-5-low)
- [Haiku 5.5 low vs GPT 5.6 Luna max](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-max-vs-haiku-5-5-low)
- [Haiku 5.5 low vs GPT 6 Luna low](https://www.stet.sh/leaderboard/compare/gpt-6-luna-low-vs-haiku-5-5-low)
- [Haiku 5.5 low vs Opus 4.8 medium](https://www.stet.sh/leaderboard/compare/haiku-5-5-low-vs-opus-4-8-medium)
- [All 153 comparisons](https://www.stet.sh/leaderboard/compare)

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-6-sol-medium-vs-haiku-5-5-low&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-6-sol-medium-vs-haiku-5-5-low). Opening a form does not execute a tool.