# GPT 6.1 Sol medium vs GPT 6 Luna max: real-repository coding test

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-gpt-6-luna-max
Source observed: 2026-11-19T11:08:43.727Z
Last checked: 2026-11-19T11:08:43.727+00:00
Indexed: 2026-11-19T11:15:39.499+00:00

---

# GPT 6.1 Sol medium vs GPT 6 Luna max

We compare two coding-agent setups on real work from the private `Stet-AI/stet` repository. GPT 6.1 Sol medium leads on agent speed, GPT 6 Luna max leads on cost, and we found no clear quality difference.

## At a glance

| Measure | Result |
|---|---|
| Quality | Even: GPT 6.1 Sol medium won 6 head-to-heads, GPT 6 Luna max won 1, and 17 tied. Ties are within judge noise. |
| Speed | GPT 6.1 Sol medium was 5.1× faster: 11.8 minutes versus 59.7 minutes per task. |
| Cost | GPT 6 Luna max was 3.0× cheaper: $0.37 versus $1.10 per task. |

## What we compared

- One real, private repository: `Stet-AI/stet`; these are not public benchmark scores.
- 24 merged changes, each run end to end by the coding agent from the repository state immediately before the change.
- The repository’s tests and judges graded each patch against the merged change.
- The sample is smaller than public benchmarks, but measures agent work on a real codebase.

[Compare GPT 6.1 Sol and GPT 6 Luna across all effort pairings](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-vs-gpt-6-luna)

Data from November 19, 2026.

## Stet scores

| Setup | Harness and reasoning | Rank | Stet score |
|---|---|---:|---:|
| GPT 6.1 Sol medium | `codex`, medium reasoning | 4 of 18 | 54.7; 95% range 46.0–63.4 |
| GPT 6 Luna max | `codex`, max reasoning | 15 of 18 | 45.3; 95% range 39.3–51.4 |

The 95% score ranges overlap. Stet score is a setup’s average chance of beating another setup on a task, from 0 to 100; 50 is even with the field.

## Quality

GPT 6.1 Sol medium won 6 head-to-head matchups, GPT 6 Luna max won 1, and 17 were tied. GPT 6.1 Sol medium’s win share was 61%, with a 95% range of 51–70%.

### Other quality measures

| Metric | GPT 6.1 Sol medium | GPT 6 Luna max |
|---|---:|---:|
| Code review pass rate | 43% (23 tasks) | 25% (24 tasks) |
| Checklist score | 0.88 (24 tasks) | 0.89 (24 tasks) |
| Equivalence rate | 43% (23 tasks) | 46% (24 tasks) |

These measures are not part of the headline Stet score and have no confidence interval. A small gap may be noise; the higher recorded value is not a proven lead.

## Task by task

GPT 6.1 Sol medium solved one task that GPT 6 Luna max did not, and GPT 6 Luna max solved one that GPT 6.1 Sol medium did not. Both solved 10; neither solved 12. On the 10 tasks both solved, judges preferred GPT 6.1 Sol medium’s patch on 4 and called 6 even.

### Only GPT 6.1 Sol medium solved

- `stet-commit-f8ae1177`: 1,312 changed lines in 9 files; judges preferred GPT 6.1 Sol medium’s patch.

### Only GPT 6 Luna max solved

- `stet-commit-f0caded9`: 113 changed lines in 4 files; judges preferred GPT 6 Luna max’s patch.

### Both solved

- `stet-43493c52`: 4 changed lines in 1 file
- `stet-3169aefa`: 6 changed lines in 1 file
- `stet-89dfbc27`: 9 changed lines in 1 file
- `stet-232b50ba`: 10 changed lines in 1 file
- `stet-5276aa3f`: 23 changed lines in 2 files
- `stet-b28f4fe7`: 29 changed lines in 1 file
- `stet-66762b4c`: 84 changed lines in 1 file; judges preferred GPT 6.1 Sol medium’s patch
- `stet-commit-5d766dde`: 218 changed lines in 2 files; judges preferred GPT 6.1 Sol medium’s patch
- `stet-commit-342d0ac0`: 640 changed lines in 8 files; judges preferred GPT 6.1 Sol medium’s patch
- `stet-commit-bbbbae09`: 941 changed lines in 7 files; judges preferred GPT 6.1 Sol medium’s patch

### Neither solved

- `stet-2450ca2d`: 2 changed lines in 1 file
- `stet-15439c21`: 5 changed lines in 2 files
- `stet-6f84e978`: 23 changed lines in 1 file
- `stet-commit-e928166f`: 149 changed lines in 2 files; judges preferred GPT 6.1 Sol medium’s patch
- `stet-commit-30506f8e`: 476 changed lines in 17 files
- `stet-commit-5e075641`: 481 changed lines in 3 files
- `stet-commit-60a0e6c2`: 552 changed lines in 11 files
- `stet-commit-a8716b88`: 564 changed lines in 10 files
- `stet-commit-87814583`: 763 changed lines in 8 files
- `stet-commit-e41c2e2d`: 977 changed lines in 5 files
- `stet-commit-d2ab0c25`: 1,236 changed lines in 16 files
- `stet-commit-2826765e`: 1,420 changed lines in 19 files

A task counts as solved when its tests passed or a judge found the fix equivalent to the merged change. Stet score also weighs code quality, so it can order setups differently from the Solved column.

## Cost and time

| Setup | Mean cost | Median cost | Mean agent time | Median agent time |
|---|---:|---:|---:|---:|
| GPT 6.1 Sol medium | $1.10 | $0.84 | 11.8 min | 11.0 min |
| GPT 6 Luna max | $0.37 | $0.29 | 59.7 min | 54.6 min |

GPT 6 Luna max cost less on all 24 tasks. GPT 6.1 Sol medium finished faster on all 24 tasks. Each task is represented on a log scale; filled marks indicate a solved task and hollow marks an unsolved task.

## Where each setup is the better choice

These are readings on smaller groups of tasks, so use them as leads to test on your own repository.

### Small fixes

10 tasks, with 2–84 changed lines in 1–2 files. Both setups solved 7/10. GPT 6 Luna max cost 5.4× less per task; GPT 6.1 Sol medium was 3.1× faster.

| | GPT 6.1 Sol medium | GPT 6 Luna max |
|---|---:|---:|
| Solved | 7/10 | 7/10 |
| Cost / task | $0.40 | $0.07 |
| Agent time / task | 5.1 min | 16.0 min |

### Large, cross-cutting changes

14 tasks, with 113–1,420 changed lines in up to 19 files. Both setups solved 4/14. GPT 6 Luna max cost 2.8× less per task; GPT 6.1 Sol medium was 5.5× faster.

| | GPT 6.1 Sol medium | GPT 6 Luna max |
|---|---:|---:|
| Solved | 4/14 | 4/14 |
| Cost / task | $1.60 | $0.58 |
| Agent time / task | 16.5 min | 91.0 min |

## Side by side

| Metric | GPT 6.1 Sol medium | GPT 6 Luna max |
|---|---|---|
| Harness | `codex` | `codex` |
| Reasoning effort | medium | max |
| Rank | 4 of 18 | 15 of 18 |
| Stet score | 54.7; 95% range 46.0–63.4 | 45.3; 95% range 39.3–51.4 |
| Solved | 11/24: 9 passed, 2 judged equivalent | 11/24: 7 passed, 4 judged equivalent |
| Cost / task | $1.10 | $0.37 |
| Agent time / task | 11.8 min | 59.7 min |

## Questions

**Which is better for coding?** We found no clear quality difference: GPT 6.1 Sol medium won 6, GPT 6 Luna max won 1, and 17 tied. GPT 6 Luna max was 3.0× cheaper; GPT 6.1 Sol medium was 5.1× faster.

**How did we test them?** Both coding agents worked on the same 24 merged changes from the private `Stet-AI/stet` repository, each from the real repository state before the change. Repository tests and judges graded every patch against the merged change; no public benchmark scores were used.

## Run the comparison on your repository

We replay your merged changes, score quality above pass/fail, and deliver comparison reports through your agent. [Get started](https://www.stet.sh/private)

## About this data

- 24 merged changes from the private `Stet-AI/stet` repository, replayed from the state immediately before each change.
- Data from November 19, 2026.
- Both setups came from leaderboard run `bmk_d4ba310fdf50db4ccb2092e3` of 18 setups.

[See all 18 setups](https://www.stet.sh/leaderboard) · [How we measure](https://www.stet.sh/methodology)

### More comparisons

- [GPT 6.1 Sol medium vs Muse Wonder Full low](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-muse-wonder-full-low)
- [GPT 6.1 Sol medium vs GPT 6 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-gpt-6-sol-medium)
- [GPT 6.1 Sol medium vs Opus 5 medium](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-opus-5-medium)
- [GPT 6.1 Sol medium vs Grok 4.7 high](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-grok-4-7-high)
- [GPT 6.1 Sol medium vs SWE-2 Max high](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-swe-2-max-high)
- [GPT 6 Luna max vs GPT 5.6 Luna max](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-max-vs-gpt-6-luna-max)
- [GPT 6 Luna max vs Haiku 5.5 low](https://www.stet.sh/leaderboard/compare/gpt-6-luna-max-vs-haiku-5-5-low)
- [GPT 6 Luna max vs Opus 4.8 medium](https://www.stet.sh/leaderboard/compare/gpt-6-luna-max-vs-opus-4-8-medium)
- [GPT 6 Luna max vs GPT 5.6 Luna low](https://www.stet.sh/leaderboard/compare/gpt-5-6-luna-low-vs-gpt-6-luna-max)
- [GPT 6 Luna max vs GLM 5.3 Flash high](https://www.stet.sh/leaderboard/compare/glm-5-3-flash-high-vs-gpt-6-luna-max)
- [All 153 comparisons](https://www.stet.sh/leaderboard/compare)

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-6-1-sol-medium-vs-gpt-6-luna-max&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fleaderboard%2Fcompare%2Fgpt-6-1-sol-medium-vs-gpt-6-luna-max). Opening a form does not execute a tool.