# GPT 5.6 Luna low vs Opus 5 medium: real-repo coding test

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-opus-5-medium
Source observed: 2026-11-19T11:08:23.855Z
Last checked: 2026-11-19T11:08:23.855+00:00
Indexed: 2026-11-19T11:18:39.246+00:00

---

# GPT 5.6 Luna low vs Opus 5 medium

GPT 5.6 Luna low led on speed and cost; we found no clear quality difference in this setup comparison. GPT 5.6 Luna low ran `codex`; Opus 5 medium ran `claude-code`. This compares setups, not models alone.

## Headline results

| Measure | Result |
|---|---|
| Quality | Even; no clear difference; GPT won 2, Opus won 9, 13 tied |
| Speed | GPT 5.6 Luna low was **3.5× faster**: 6.7 min vs 23.2 min per task |
| Cost | GPT 5.6 Luna low was **133× cheaper**: $0.05 vs $6.78 per task |

## What this comparison is

- **1 real repository:** private `Stet-AI/stet`; no public benchmark scores.
- **24 merged changes:** each setup ran end to end from the repository state before the change.
- **Graded against the merged change:** repository tests and judges compared each patch with the merged result.
- **Quality, cost, and time:** a smaller sample than public benchmarks, but real agent work on a real codebase.

- [GPT 5.6 Luna vs Opus 5: all effort pairings](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-vs-opus-5)

Data: **November 19, 2026**.

## Stet score

| Setup | Harness and reasoning | Rank | Score and 95% range |
|---|---|---:|---|
| GPT 5.6 Luna low | `codex` · low reasoning | 17 of 18 | 43.5; 36.7–50.2 |
| Opus 5 medium | `claude-code` · medium reasoning | 2 of 18 | 61.4; 52.6–70.3 |

The two 95% score ranges do not overlap. Opus 5 medium’s win share was 66%, with a 95% range of 54–77%. Ties are within judge noise.

## Other quality measures

| Metric | GPT 5.6 Luna low | Opus 5 medium |
|---|---:|---:|
| Code review pass rate | 29% · 24 tasks | 38% · 24 tasks |
| Checklist score | 0.60 · 24 tasks | 0.89 · 24 tasks |
| Equivalence rate | 38% · 24 tasks | 54% · 24 tasks |

These measures are not part of the headline Stet score and have no confidence interval. A small gap may be noise; the marked side has the higher recorded value, not a proven lead.

## Task by task

GPT solved 0 tasks that Opus did not, and Opus solved 5 that GPT did not. Both solved 10; neither solved 9. On the 10 tasks both solved, judges preferred Opus’s patch on 3, GPT’s on 2, and called 5 even.

### Only GPT 5.6 Luna low solved — 0

None.

### Only Opus 5 medium solved — 5

- `stet-2450ca2d`: 2 changed lines in 1 file; judges preferred Opus’s patch.
- `stet-6f84e978`: 23 changed lines in 1 file; judges preferred Opus’s patch.
- `stet-commit-bbbbae09`: 941 changed lines in 7 files; judges preferred Opus’s patch.
- `stet-commit-f8ae1177`: 1,312 changed lines in 9 files; judges preferred Opus’s patch.
- `stet-commit-2826765e`: 1,420 changed lines in 19 files; judges preferred Opus’s patch.

### Both solved — 10

- `stet-43493c52`: 4 changed lines in 1 file; judges preferred GPT’s patch.
- `stet-15439c21`: 5 changed lines in 2 files.
- `stet-3169aefa`: 6 changed lines in 1 file.
- `stet-89dfbc27`: 9 changed lines in 1 file.
- `stet-232b50ba`: 10 changed lines in 1 file.
- `stet-5276aa3f`: 23 changed lines in 2 files.
- `stet-b28f4fe7`: 29 changed lines in 1 file; judges preferred GPT’s patch.
- `stet-66762b4c`: 84 changed lines in 1 file; judges preferred Opus’s patch.
- `stet-commit-f0caded9`: 113 changed lines in 4 files; judges preferred Opus’s patch.
- `stet-commit-5d766dde`: 218 changed lines in 2 files; judges preferred Opus’s patch.

### Neither solved — 9

- `stet-commit-e928166f`: 149 changed lines in 2 files; judges preferred Opus’s patch.
- `stet-commit-30506f8e`: 476 changed lines in 17 files.
- `stet-commit-5e075641`: 481 changed lines in 3 files.
- `stet-commit-60a0e6c2`: 552 changed lines in 11 files.
- `stet-commit-a8716b88`: 564 changed lines in 10 files.
- `stet-commit-342d0ac0`: 640 changed lines in 8 files.
- `stet-commit-87814583`: 763 changed lines in 8 files.
- `stet-commit-e41c2e2d`: 977 changed lines in 5 files.
- `stet-commit-d2ab0c25`: 1,236 changed lines in 16 files.

A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change. Stet score also weighs code quality, so it can order setups differently.

## All 24 compared tasks

| Task | GPT 5.6 Luna low | Opus 5 medium | Solved by | Better patch |
|---|---|---|---|---|
| `stet-89dfbc27` · 9 lines, 1 file | Passed tests · $0.04 · 6.5 min | Passed tests · $1.53 · 32.6 min | Both | Even |
| `stet-b28f4fe7` · 29 lines, 1 file | Passed tests · $0.03 · 2.6 min | Passed tests · $0.69 · 19.4 min | Both | GPT |
| `stet-232b50ba` · 10 lines, 1 file | Passed tests · $0.04 · 14.5 min | Passed tests · $1.21 · 7.9 min | Both | Even |
| `stet-3169aefa` · 6 lines, 1 file | Passed tests · $0.02 · 4.9 min | Passed tests · $1.09 · 9.2 min | Both | Even |
| `stet-5276aa3f` · 23 lines, 2 files | Passed tests · $0.05 · 13.4 min | Passed tests · $1.03 · 7.7 min | Both | Even |
| `stet-66762b4c` · 84 lines, 1 file | Equivalent to merged fix · $0.02 · 3.7 min | Passed tests · $1.74 · 36.7 min | Both | Opus |
| `stet-commit-bbbbae09` · 941 lines, 7 files | Failed · $0.04 · 6.6 min | Passed tests · $8.24 · 18.7 min | Opus only | Opus |
| `stet-commit-5d766dde` · 218 lines, 2 files | Equivalent to merged fix · $0.05 · 10.7 min | Equivalent to merged fix · $3.36 · 11.0 min | Both | Opus |
| `stet-43493c52` · 4 lines, 1 file | Passed tests · $0.03 · 4.0 min | Passed tests · $0.52 · 2.3 min | Both | GPT |
| `stet-commit-f0caded9` · 113 lines, 4 files | Equivalent to merged fix · $0.04 · 2.9 min | Passed tests · $3.43 · 10.8 min | Both | Opus |
| `stet-15439c21` · 5 lines, 2 files | Passed tests · $0.03 · 9.5 min | Passed tests · $0.53 · 4.9 min | Both | Even |
| `stet-commit-f8ae1177` · 1,312 lines, 9 files | Failed · $0.05 · 6.7 min | Passed tests · $18.97 · 27.5 min | Opus only | Opus |
| `stet-2450ca2d` · 2 lines, 1 file | Failed · $0.02 · 4.2 min | Passed tests · $0.82 · 21.7 min | Opus only | Opus |
| `stet-commit-342d0ac0` · 640 lines, 8 files | Failed · $0.12 · 12.4 min | Failed · $5.56 · 16.4 min | Neither | Even |
| `stet-6f84e978` · 23 lines, 1 file | Failed · $0.03 · 6.9 min | Passed tests · $1.39 · 34.9 min | Opus only | Opus |
| `stet-commit-5e075641` · 481 lines, 3 files | Failed · $0.07 · 5.7 min | Failed · $10.93 · 31.1 min | Neither | Even |
| `stet-commit-60a0e6c2` · 552 lines, 11 files | Failed · $0.04 · 2.6 min | Failed · $7.75 · 24.8 min | Neither | Even |
| `stet-commit-2826765e` · 1,420 lines, 19 files | Failed · $0.04 · 7.2 min | Equivalent to merged fix · $17.19 · 30.1 min | Opus only | Opus |
| `stet-commit-a8716b88` · 564 lines, 10 files | Failed · $0.10 · 5.8 min | Failed · $12.40 · 43.1 min | Neither | Even |
| `stet-commit-e928166f` · 149 lines, 2 files | Failed · $0.06 · 3.4 min | Failed · $5.94 · 25.2 min | Neither | Opus |
| `stet-commit-30506f8e` · 476 lines, 17 files | Failed · $0.12 · 10.1 min | Failed · $15.36 · 43.4 min | Neither | Even |
| `stet-commit-87814583` · 763 lines, 8 files | Failed · $0.03 · 2.7 min | Failed · $14.95 · 33.3 min | Neither | Even |
| `stet-commit-d2ab0c25` · 1,236 lines, 16 files | Failed · $0.05 · 5.0 min | Failed · $15.17 · 25.8 min | Neither | Even |
| `stet-commit-e41c2e2d` · 977 lines, 5 files | Failed · $0.10 · 8.3 min | Failed · $12.80 · 38.3 min | Neither | Even |

## Cost and time on each task

- **Cost:** GPT mean $0.05, median $0.04; Opus mean $6.78, median $4.50. GPT cost less on all 24 tasks.
- **Agent time:** GPT mean 6.7 min, median 6.2 min; Opus mean 23.2 min, median 25.0 min. GPT finished faster on 20 of 24 tasks; Opus on 4.

## Where each setup is the better choice

These are readings on smaller task groups and are leads to test on your repository.

### Small fixes

10 tasks, with 2–84 changed lines in 1–2 files. Opus solved 10/10 versus GPT’s 8/10; GPT cost 34× less per task and was 2.5× faster.

| Metric | GPT 5.6 Luna low | Opus 5 medium |
|---|---:|---:|
| Solved | 8/10 | 10/10 |
| Cost / task | $0.03 | $1.05 |
| Agent time / task | 7.0 min | 17.7 min |

### Large, cross-cutting changes

14 tasks, with 113–1,420 changed lines in up to 19 files. Opus solved 5/14 versus GPT’s 2/14; GPT cost 167× less per task and was 4.2× faster.

| Metric | GPT 5.6 Luna low | Opus 5 medium |
|---|---:|---:|
| Solved | 2/14 | 5/14 |
| Cost / task | $0.07 | $10.86 |
| Agent time / task | 6.4 min | 27.1 min |

## Side by side

| Metric | GPT 5.6 Luna low | Opus 5 medium |
|---|---|---|
| Harness | `codex` | `claude-code` |
| Reasoning effort | low | medium |
| Rank | 17 of 18 | 2 of 18 |
| Stet score | 43.5; 95% range 36.7–50.2 | 61.4; 95% range 52.6–70.3 |
| Solved | 10/24: 7 passed, 3 judged equivalent | 15/24: 13 passed, 2 judged equivalent |
| Cost / task | $0.05 | $6.78 |
| Agent time / task | 6.7 min | 23.2 min |

## Research on these models

- [Cost per successful task ranked the coding agent that lost every head to head 1st](https://www.stet.sh/blog/cost-per-successful-task)
- [A benchmark mined from my own repo agreed with FrontierCode](https://www.stet.sh/blog/benchmark-agreed-with-frontier-code)
- [On 25 Stet Tasks, Opus 4.8 Wrote Smaller Patches. Opus 5 Searched Wider.](https://www.stet.sh/blog/opus-4-8-vs-opus-5-same-score-different-routes)

## Questions

**Which is better for coding?** We found no clear quality difference: Opus won 9, GPT won 2, and 13 tied. GPT led on cost at 133× cheaper and agent time at 3.5× faster. The measurement used 24 merged changes from `Stet-AI/stet`.

**How did we test them?** Both agents worked on the same 24 merged changes in the private repository, starting from the repository state before each change. Repository tests and judges graded each patch against the merged change; no public benchmark scores were used. A Stet score is a setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field.

## See which setup wins on your repo

We replay your merged changes, score quality above pass/fail, and deliver comparison reports through your agent. [Get started](https://www.stet.sh/private)

## About this data

- 24 tasks from the private `Stet-AI/stet` repository, replayed from the repository state just before each change.
- Data from November 19, 2026.
- Both setups came from the same leaderboard run of 18 setups: `bmk_d4ba310fdf50db4ccb2092e3`.

- [See all 18 setups](https://www.stet.sh/leaderboard)
- [How Stet measures](https://www.stet.sh/methodology)

## More comparisons

- [GPT 5.6 Luna low vs Haiku 5.5 low](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-haiku-5-5-low)
- [GPT 5.6 Luna low vs GPT 6 Luna low](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-gpt-6-luna-low)
- [GPT 5.6 Luna low vs GPT 6 Luna max](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-gpt-6-luna-max)
- [GPT 5.6 Luna low vs GPT 5.6 Luna max](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-gpt-5-6-luna-max)
- [GPT 5.6 Luna low vs Opus 4.8 medium](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-opus-4-8-medium)
- [Opus 5 medium vs SWE-2 Max high](https://www.stet.sh/leaderboard/compare/opus-5-medium-vs-swe-2-max-high)
- [Opus 5 medium vs Muse Wonder Full low](https://www.stet.sh/leaderboard/compare/muse-wonder-full-low-vs-opus-5-medium)
- [Opus 5 medium vs GPT 6.1 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-opus-5-medium)
- [Opus 5 medium vs GPT 6 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-sol-medium-vs-opus-5-medium)
- [Opus 5 medium vs Grok 4.7 high](https://www.stet.sh/leaderboard/compare/grok-4-7-high-vs-opus-5-medium)
- [All 153 comparisons](https://www.stet.sh/leaderboard/compare)

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-5-6-luna-low-vs-opus-5-medium&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-5-6-luna-low-vs-opus-5-medium). Opening a form does not execute a tool.