# GPT 5.6 Luna max vs Opus 4.8 medium

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-max-vs-opus-4-8-medium
Source observed: 2026-11-19T11:08:22.825Z
Last checked: 2026-11-19T11:08:22.825+00:00
Indexed: 2026-11-19T11:18:40.346+00:00

---

# GPT 5.6 Luna max vs Opus 4.8 medium

We compare GPT 5.6 Luna max and Opus 4.8 medium as complete coding-agent setups on 24 merged changes from our private `Stet-AI/stet` repository. GPT 5.6 Luna max used **codex** with **max** reasoning; Opus 4.8 medium used **claude-code** with **medium** reasoning.

## Summary

- **Quality:** Even/no clear difference: GPT won 4, Opus won 4, 14 were tied, and 2 were undecided.
- **Speed:** Opus was **1.2× faster**: 23.3 versus 28.5 minutes per task.
- **Cost:** GPT was **8.5× cheaper**: $0.90 versus $7.68 per task.

## Test scope and scores

Each setup ran end to end from the repository state before each change. Repository tests and judges compared patches with merged changes. Data: **November 19, 2026**. Both setups came from leaderboard run `bmk_d4ba310fdf50db4ccb2092e3`, containing 18 setups.

| Setup | Harness | Reasoning | Rank | Stet score | 95% range |
|---|---|---:|---:|---:|---:|
| GPT 5.6 Luna max | codex | max | 14 of 18 | 46.7 | 40.6–52.6 |
| Opus 4.8 medium | claude-code | medium | 13 of 18 | 46.8 | 38.0–55.7 |

The score ranges overlap. Stet score is a setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field.

## Quality

GPT won 4 head-to-head matchups, Opus won 4, 14 were tied, and 2 were undecided. GPT’s win share was 51%, with a 95% range of 38–63%. Ties are within judge noise.

| Metric | GPT 5.6 Luna max | Opus 4.8 medium |
|---|---:|---:|
| Code review pass rate | 38% (24 tasks) | 33% (24 tasks) |
| Checklist score | 0.86 (24 tasks) | 0.78 (24 tasks) |
| Equivalence rate | 38% (24 tasks) | 46% (24 tasks) |

These additional measures are not part of the headline Stet score and have no confidence interval.

## Task by task

GPT solved 0 tasks Opus did not; Opus solved 3 GPT did not. Both solved 10 and neither solved 9. On the 10 tasks both solved, judges preferred GPT’s patch on 4, Opus’s on 1, and called 5 even.

### Only GPT 5.6 Luna max solved

None.

### Only Opus 4.8 medium solved

- `stet-15439c21` — 5 changed lines in 2 files; judges preferred Opus’s patch.
- `stet-6f84e978` — 23 changed lines in 1 file; judges preferred Opus’s patch.
- `stet-commit-f8ae1177` — 1,312 changed lines in 9 files; judges preferred Opus’s patch.

### Both solved

- `stet-43493c52` — 4 changed lines in 1 file.
- `stet-3169aefa` — 6 changed lines in 1 file.
- `stet-89dfbc27` — 9 changed lines in 1 file; judges preferred GPT’s patch.
- `stet-232b50ba` — 10 changed lines in 1 file; judges preferred GPT’s patch.
- `stet-5276aa3f` — 23 changed lines in 2 files.
- `stet-b28f4fe7` — 29 changed lines in 1 file; judges preferred GPT’s patch.
- `stet-66762b4c` — 84 changed lines in 1 file; judges preferred Opus’s patch.
- `stet-commit-f0caded9` — 113 changed lines in 4 files.
- `stet-commit-5d766dde` — 218 changed lines in 2 files.
- `stet-commit-bbbbae09` — 941 changed lines in 7 files; judges preferred GPT’s patch.

### Neither solved

- `stet-2450ca2d` — 2 changed lines in 1 file.
- `stet-commit-e928166f` — 149 changed lines in 2 files.
- `stet-commit-30506f8e` — 476 changed lines in 17 files.
- `stet-commit-5e075641` — 481 changed lines in 3 files.
- `stet-commit-a8716b88` — 564 changed lines in 10 files.
- `stet-commit-342d0ac0` — 640 changed lines in 8 files.
- `stet-commit-87814583` — 763 changed lines in 8 files.
- `stet-commit-e41c2e2d` — 977 changed lines in 5 files.
- `stet-commit-2826765e` — 1,420 changed lines in 19 files.

Two of 24 tasks are left out because one setup has no scored result. A task counts as solved when tests pass or a judge finds the fix equivalent to the merged change. Stet score also weighs code quality.

## Cost and time

GPT cost less on all 22 compared tasks, with mean $0.88 and median $0.73. Opus averaged $7.46 with a $3.18 median. Opus finished faster on 15 tasks; GPT was faster on 7. Mean agent time was 28.2 minutes for GPT and 23.8 for Opus; medians were 22.3 and 17.3 minutes.

## Where each setup is the better choice

### Small fixes

Ten tasks covered 2–84 changed lines in 1–2 files. Opus solved 9/10 versus GPT’s 7/10. GPT cost 5.2× less and was 1.2× faster.

| | GPT | Opus |
|---|---:|---:|
| Solved | 7/10 | 9/10 |
| Cost / task | $0.23 | $1.22 |
| Agent time / task | 9.8 min | 11.6 min |

### Large, cross-cutting changes

Fourteen tasks covered 113–1,420 changed lines in up to 19 files. Opus solved 4/12 versus GPT’s 3/12. GPT cost 8.9× less; Opus was 1.3× faster.

| | GPT | Opus |
|---|---:|---:|
| Solved | 3/12 | 4/12 |
| Cost / task | $1.43 | $12.66 |
| Agent time / task | 43.6 min | 34.0 min |

## Side by side

| Metric | GPT 5.6 Luna max | Opus 4.8 medium |
|---|---|---|
| Harness | codex | claude-code |
| Reasoning effort | max | medium |
| Rank | 14 of 18 | 13 of 18 |
| Stet score | 46.7; 95% range 40.6–52.6 | 46.8; 95% range 38.0–55.7 |
| Solved | 10/24: 7 passed, 3 judged equivalent | 13/24: 11 passed, 2 judged equivalent |
| Cost / task | $0.90 | $7.68 |
| Agent time / task | 28.5 min | 23.3 min |

## Questions

**Which is better for coding?** We found no clear quality difference: each won 4, 14 tied, and 2 were undecided. GPT was 8.5× cheaper; Opus was 1.2× faster.

**How were they tested?** Both agents worked on the same 24 merged changes from the private repository, from the real state before each change. Repository tests and judges graded each patch against the merged change; no public benchmark scores were used.

## See which setup wins on your repo

We replay your merged changes, score quality above pass/fail, and deliver comparison reports through your agent. [Get started](https://www.stet.sh/private)

## Research links

- [Cost per successful task ranked the coding agent that lost every head to head 1st](https://www.stet.sh/blog/cost-per-successful-task)
- [GLM 5.2 vs Composer 2.5 on 50 real Go and Rust PRs: last on quality, and not the cheapest](https://www.stet.sh/blog/glm-5-2-passes-tests-fails-review)
- [Composer 2.5 vs Opus 4.8 vs GPT-5.5 vs Opus 4.7 - 50 Real PRs in Go and Rust](https://www.stet.sh/blog/opus-48-vs-gpt-55-vs-opus-47-vs-composer-25)
- [See all 18 setups](https://www.stet.sh/leaderboard)
- [How Stet measures](https://www.stet.sh/methodology)

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-5-6-luna-max-vs-opus-4-8-medium&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-5-6-luna-max-vs-opus-4-8-medium). Opening a form does not execute a tool.