# GPT 5.6 Luna low vs Opus 4.8 medium: real-repository coding test

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-opus-4-8-medium
Source observed: 2026-11-19T11:08:40.787Z
Last checked: 2026-11-19T11:08:40.787+00:00
Indexed: 2026-11-19T11:16:17.007+00:00

---

# GPT 5.6 Luna low vs Opus 4.8 medium

We compare two coding-agent setups on real work from our private `Stet-AI/stet` repository. GPT 5.6 Luna low ran with **codex** and low reasoning effort; Opus 4.8 medium ran with **claude-code** and medium reasoning effort. This compares the complete setups, not the models alone.

## Headline results

| Measure | Result |
| --- | --- |
| Quality | **Even**: GPT 5.6 Luna low won 4 head-to-head tasks, Opus 4.8 medium won 4, 14 were tied, and 2 were undecided. |
| Speed | GPT 5.6 Luna low was **3.5× faster**: 6.7 minutes versus 23.3 minutes per task. |
| Cost | GPT 5.6 Luna low was **151× cheaper**: $0.05 versus $7.68 per task. |

## What we compared

- **1 real repository:** private `Stet-AI/stet`; these are not public benchmark scores.
- **24 merged changes:** each coding agent ran end to end from the repository state before the change.
- **Evaluation against the merged change:** we used the repository’s tests and judges comparing each patch with the merged change.
- **Quality, cost, and time:** the sample is smaller than public benchmarks, but represents agent work on a real codebase.

[GPT 5.6 Luna vs Opus 4.8: all effort pairings](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-vs-opus-4-8)

**24 tasks · Data from November 19, 2026**

## Stet score

Our Stet score represents a setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field. The two 95% score ranges overlap.

| Setup | Harness and reasoning | Rank | Stet score |
| --- | --- | --- | --- |
| GPT 5.6 Luna low | codex · low reasoning | 17 of 18 | 43.5; 95% range 36.7–50.2 |
| Opus 4.8 medium | claude-code · medium reasoning | 13 of 18 | 46.8; 95% range 38.0–55.7 |

## Quality

### Head-to-head matchups

- GPT 5.6 Luna low won **4**.
- The setups tied on **14**.
- **2** were undecided.
- Opus 4.8 medium won **4**.

Opus 4.8 medium’s win share was 52%, with a 95% range of 40–64%. Ties are within judge noise.

### Other quality measures

| Metric | GPT 5.6 Luna low | Opus 4.8 medium |
| --- | ---: | ---: |
| Code review pass rate | 29% · 24 tasks | 33% · 24 tasks |
| Checklist score | 0.60 · 24 tasks | 0.78 · 24 tasks |
| Equivalence rate | 38% · 24 tasks | 46% · 24 tasks |

These measures are not part of the headline Stet score and have no confidence interval. A small gap may be noise; the higher recorded value is not a proven lead.

## Task by task

GPT 5.6 Luna low solved no tasks that Opus 4.8 medium failed to solve. Opus 4.8 medium solved 3 tasks that GPT 5.6 Luna low did not. Both solved 10 tasks, and neither solved 9. Among the 10 tasks both solved, judges preferred GPT 5.6 Luna low’s patch on 4, Opus 4.8 medium’s patch on 1, and called 5 even.

### Only GPT 5.6 Luna low solved — 0 tasks

None.

### Only Opus 4.8 medium solved — 3 tasks

- `stet-6f84e978`: 23 changed lines in 1 file; judges preferred Opus 4.8 medium’s patch.
- `stet-commit-bbbbae09`: 941 changed lines in 7 files; judges preferred Opus 4.8 medium’s patch.
- `stet-commit-f8ae1177`: 1,312 changed lines in 9 files; judges preferred Opus 4.8 medium’s patch.

### Both solved — 10 tasks

- `stet-43493c52`: 4 changed lines in 1 file; judges preferred GPT 5.6 Luna low’s patch.
- `stet-15439c21`: 5 changed lines in 2 files.
- `stet-3169aefa`: 6 changed lines in 1 file.
- `stet-89dfbc27`: 9 changed lines in 1 file; judges preferred GPT 5.6 Luna low’s patch.
- `stet-232b50ba`: 10 changed lines in 1 file; judges preferred GPT 5.6 Luna low’s patch.
- `stet-5276aa3f`: 23 changed lines in 2 files.
- `stet-b28f4fe7`: 29 changed lines in 1 file; judges preferred GPT 5.6 Luna low’s patch.
- `stet-66762b4c`: 84 changed lines in 1 file; judges preferred Opus 4.8 medium’s patch.
- `stet-commit-f0caded9`: 113 changed lines in 4 files.
- `stet-commit-5d766dde`: 218 changed lines in 2 files.

### Neither solved — 9 tasks

- `stet-2450ca2d`: 2 changed lines in 1 file.
- `stet-commit-e928166f`: 149 changed lines in 2 files.
- `stet-commit-30506f8e`: 476 changed lines in 17 files.
- `stet-commit-5e075641`: 481 changed lines in 3 files.
- `stet-commit-a8716b88`: 564 changed lines in 10 files.
- `stet-commit-342d0ac0`: 640 changed lines in 8 files.
- `stet-commit-87814583`: 763 changed lines in 8 files.
- `stet-commit-e41c2e2d`: 977 changed lines in 5 files.
- `stet-commit-2826765e`: 1,420 changed lines in 19 files.

The task-size bars use changed lines in the merged change on a logarithmic scale. A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change, as in the Solved column. The Stet score also weighs code quality, so it can order the setups differently. Two of the 24 tasks are excluded because one setup has no scored result for each of them.

### All 22 compared tasks

| Task | GPT 5.6 Luna low | Opus 4.8 medium | Solved by | Better patch |
| --- | --- | --- | --- | --- |
| `stet-89dfbc27` · 9 lines, 1 file | Passed tests · $0.04 · 6.5 min | Passed tests · $0.83 · 11.6 min | Both | GPT 5.6 Luna low |
| `stet-b28f4fe7` · 29 lines, 1 file | Passed tests · $0.03 · 2.6 min | Passed tests · $1.09 · 18.0 min | Both | GPT 5.6 Luna low |
| `stet-232b50ba` · 10 lines, 1 file | Passed tests · $0.04 · 14.5 min | Passed tests · $1.43 · 9.1 min | Both | GPT 5.6 Luna low |
| `stet-3169aefa` · 6 lines, 1 file | Passed tests · $0.02 · 4.9 min | Passed tests · $1.06 · 2.6 min | Both | Even |
| `stet-5276aa3f` · 23 lines, 2 files | Passed tests · $0.05 · 13.4 min | Passed tests · $1.15 · 6.7 min | Both | Even |
| `stet-66762b4c` · 84 lines, 1 file | Equivalent to the merged fix · $0.02 · 3.7 min | Passed tests · $1.94 · 26.5 min | Both | Opus 4.8 medium |
| `stet-commit-bbbbae09` · 941 lines, 7 files | Failed · $0.04 · 6.6 min | Passed tests · $12.48 · 29.1 min | Opus 4.8 medium only | Opus 4.8 medium |
| `stet-commit-5d766dde` · 218 lines, 2 files | Equivalent to the merged fix · $0.05 · 10.7 min | Equivalent to the merged fix · $3.10 · 9.9 min | Both | Even |
| `stet-43493c52` · 4 lines, 1 file | Passed tests · $0.03 · 4.0 min | Passed tests · $0.85 · 4.3 min | Both | GPT 5.6 Luna low |
| `stet-commit-f0caded9` · 113 lines, 4 files | Equivalent to the merged fix · $0.04 · 2.9 min | Equivalent to the merged fix · $4.29 · 39.9 min | Both | Even |
| `stet-15439c21` · 5 lines, 2 files | Passed tests · $0.03 · 9.5 min | Passed tests · $1.57 · 16.3 min | Both | Even |
| `stet-commit-f8ae1177` · 1,312 lines, 9 files | Failed · $0.05 · 6.7 min | Passed tests · $28.54 · 45.5 min | Opus 4.8 medium only | Opus 4.8 medium |
| `stet-2450ca2d` · 2 lines, 1 file | Failed · $0.02 · 4.2 min | Failed · $0.92 · 14.4 min | Neither | Even |
| `stet-commit-342d0ac0` · 640 lines, 8 files | Failed · $0.12 · 12.4 min | Failed · $10.51 · 60.2 min | Neither | Even |
| `stet-6f84e978` · 23 lines, 1 file | Failed · $0.03 · 6.9 min | Passed tests · $1.35 · 6.1 min | Opus 4.8 medium only | Opus 4.8 medium |
| `stet-commit-5e075641` · 481 lines, 3 files | Failed · $0.07 · 5.7 min | Failed · $13.27 · 28.8 min | Neither | Even |
| `stet-commit-2826765e` · 1,420 lines, 19 files | Failed · $0.04 · 7.2 min | Failed · $18.37 · 63.2 min | Neither | Even |
| `stet-commit-a8716b88` · 564 lines, 10 files | Failed · $0.10 · 5.8 min | Failed · $26.24 · 38.3 min | Neither | Even |
| `stet-commit-e928166f` · 149 lines, 2 files | Failed · $0.06 · 3.4 min | Failed · $3.27 · 24.9 min | Neither | Even |
| `stet-commit-30506f8e` · 476 lines, 17 files | Failed · $0.12 · 10.1 min | Failed · $19.23 · 38.0 min | Neither | Even |
| `stet-commit-87814583` · 763 lines, 8 files | Failed · $0.03 · 2.7 min | Failed · $7.94 · 16.5 min | Neither | Even |
| `stet-commit-e41c2e2d` · 977 lines, 5 files | Failed · $0.10 · 8.3 min | Failed · $4.71 · 13.9 min | Neither | Even |

## Cost and time on each task

Each dot represents one task on a logarithmic scale. Filled dots indicate a solved task; hollow dots indicate an unsolved task. The line marks the median.

- **Cost per task:** GPT 5.6 Luna low mean **$0.05**, median **$0.04**; Opus 4.8 medium mean **$7.46**, median **$3.18**. GPT 5.6 Luna low cost less on all 22 tasks.
- **Agent time per task:** GPT 5.6 Luna low mean **6.9 min**, median **6.5 min**; Opus 4.8 medium mean **23.8 min**, median **17.3 min**. GPT 5.6 Luna low finished faster on 17 of 22 tasks; Opus 4.8 medium was faster on 5.

## Where each setup is the better choice

These are readings for smaller task groups than the overall standings, so we recommend treating them as leads to test on your own repository.

### Small fixes

**10 tasks · 2–84 changed lines in 1–2 files**

Opus 4.8 medium solved one more task, 9 of 10 versus 8 of 10. GPT 5.6 Luna low cost 39× less per task and was 1.6× faster.

| Measure | GPT 5.6 Luna low | Opus 4.8 medium |
| --- | ---: | ---: |
| Solved | 8/10 | 9/10 |
| Cost per task | $0.03 | $1.22 |
| Agent time per task | 7.0 min | 11.6 min |

### Large, cross-cutting changes

**14 tasks · 113–1,420 changed lines in up to 19 files**

Opus 4.8 medium solved two more tasks, 4 of 12 versus 2 of 12. GPT 5.6 Luna low cost 184× less per task and was 4.9× faster.

| Measure | GPT 5.6 Luna low | Opus 4.8 medium |
| --- | ---: | ---: |
| Solved | 2/12 | 4/12 |
| Cost per task | $0.07 | $12.66 |
| Agent time per task | 6.9 min | 34.0 min |

## Side by side

| Metric | GPT 5.6 Luna low | Opus 4.8 medium |
| --- | --- | --- |
| Harness | codex | claude-code |
| Reasoning effort | low | medium |
| Rank | 17 of 18 | 13 of 18 |
| Stet score | 43.5; 95% range 36.7–50.2 | 46.8; 95% range 38.0–55.7 |
| Solved | 10/24: 7 passed, 3 judged equivalent | 13/24: 11 passed, 2 judged equivalent |
| Cost per task | $0.05 | $7.68 |
| Agent time per task | 6.7 min | 23.3 min |

## Research on these models

- [Cost per successful task ranked the coding agent that lost every head to head 1st](https://www.stet.sh/blog/cost-per-successful-task)
- [GLM 5.2 vs Composer 2.5 on 50 real Go and Rust PRs: last on quality, and not the cheapest](https://www.stet.sh/blog/glm-5-2-passes-tests-fails-review)
- [Composer 2.5 vs Opus 4.8 vs GPT-5.5 vs Opus 4.7 - 50 Real PRs in Go and Rust](https://www.stet.sh/blog/opus-48-vs-gpt-55-vs-opus-47-vs-composer-25)

## Questions

### Which is better for coding, GPT 5.6 Luna low or Opus 4.8 medium?

We found no clear quality difference: GPT 5.6 Luna low won 4, Opus 4.8 medium won 4, 14 tied, and 2 were undecided. GPT 5.6 Luna low led on cost and agent time: 151× cheaper and 3.5× faster. We measured this on 24 merged changes from `Stet-AI/stet`.

### How were GPT 5.6 Luna low and Opus 4.8 medium tested?

Both coding agents worked on the same 24 merged changes from our private `Stet-AI/stet` repository, each in the real repository state from before the change. The repository’s tests and judges graded every patch against the merged change. We used no public benchmark scores.

## Test your own repository

We replay your merged changes, score quality above pass/fail, and deliver comparison reports through your agent.

[Get started](https://www.stet.sh/private)

## About this data

- 24 tasks, each a merged change from the private `Stet-AI/stet` repository replayed from the repository state immediately before the change.
- Data from **November 19, 2026**.
- Both setups came from the same leaderboard run of 18 setups: `bmk_d4ba310fdf50db4ccb2092e3`.

[See all 18 setups on the leaderboard](https://www.stet.sh/leaderboard) · [How Stet measures](https://www.stet.sh/methodology)

## More comparisons

- [GPT 5.6 Luna low vs Haiku 5.5 low](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-haiku-5-5-low)
- [GPT 5.6 Luna low vs GPT 6 Luna low](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-gpt-6-luna-low)
- [GPT 5.6 Luna low vs GPT 6 Luna max](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-gpt-6-luna-max)
- [GPT 5.6 Luna low vs GPT 5.6 Luna max](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-gpt-5-6-luna-max)
- [GPT 5.6 Luna low vs GLM 5.3 Flash high](https://www.stet.sh/leaderboard/compare/glm-5-3-flash-high-vs-gpt-5-6-luna-low)
- [Opus 4.8 medium vs GLM 5.3 Flash high](https://www.stet.sh/leaderboard/compare/glm-5-3-flash-high-vs-opus-4-8-medium)
- [Opus 4.8 medium vs GPT 5.6 Luna max](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-max-vs-opus-4-8-medium)
- [Opus 4.8 medium vs Muse Spark 1.3 high](https://www.stet.sh/leaderboard/compare/muse-spark-1-3-high-vs-opus-4-8-medium)
- [Opus 4.8 medium vs GPT 6 Luna max](https://www.stet.sh/leaderboard/compare/gpt-6-luna-max-vs-opus-4-8-medium)
- [Opus 4.8 medium vs Haiku 5.5 max](https://www.stet.sh/leaderboard/compare/haiku-5-5-max-vs-opus-4-8-medium)
- [All 153 comparisons](https://www.stet.sh/leaderboard/compare)

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-5-6-luna-low-vs-opus-4-8-medium&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-5-6-luna-low-vs-opus-4-8-medium). Opening a form does not execute a tool.