# GPT 6.1 Sol vs GPT 6 Sol: cost and patch tradeoffs

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/blog/gpt-6-1-sol-vs-gpt-6-sol
Source observed: 2026-11-08T19:52:03.726Z
Last checked: 2026-11-08T19:52:03.726+00:00
Indexed: 2026-11-08T20:38:38.208+00:00

---

# GPT 6.1 Sol vs GPT 6 Sol: cost and patch tradeoffs

**October 12, 2026 · 5 min read** · [Ben, Founder, Stet](https://www.stet.sh/about)

We use Stet to evaluate coding agents against changes merged into a repository. In this study, we ran GPT 6.1 Sol and GPT 6 Sol in Codex at medium reasoning on 24 tasks from a small-to-medium Go and TypeScript codebase, with one attempt per task.

## TL;DR

GPT 6.1 Sol is a cheaper replacement that preserved comparable quality while producing larger patches. It won 9 tasks, tied 10, and lost 5 head to head; cost less on 22 of 24 tasks; had a median relative task cost of **0.56×**; and ranked **2nd of 14** on our leaderboard. GPT 6 Sol ranked 7th, while Opus 5 ranked first.

- GPT 6.1 Sol’s typical task cost was about half as much and took about as long.
- GPT 6.1 Sol’s median patch was 303 changed lines versus 197 for GPT 6 Sol.
- Checklist and code-review means favored GPT 6.1 Sol, while correctness was close and slightly favored GPT 6 Sol.

## How we grade

We build each task from a merged change in a private repository: the repository snapshot immediately before the merge, the task prompt, and tests that the agent never sees. We prevent the agent from fetching the result through GitHub.

We combine four signals:

- **Correctness:** original tests pass or fail, or the patch is equivalent to the merged change.
- **Checklist:** required task behaviors are judged item by item. A model writes the checklist from the prompt, merged change, and tests without seeing any agent patch; the same judge grades every model. The judge used here was GPT 5.6 Terra.
- **Code review:** code reuse, abstraction levels, test style, unused code, guards, and scope—whether a senior engineer would want to merge the patch.
- **Cost and time:** recorded token usage, coding cost, and elapsed minutes.

Correctness contributes 53% of the combined patch score; checklist and review contribute 47%. We compare the models task by task, with close scores counted as ties.

### Replay and repair example

The interactive study includes a task about preserving provenance during test-command replay and validation repair. The task requires recoverable failures to be retried with recorded attempt history, replay of only the actual test invocation, and repaired grades tied to their original measurement context. Unsafe or ambiguous commands and unrecoverable failures must be rejected rather than guessed through.

The displayed task prompt includes these required interfaces and fields:

```text
AttemptCount int `json:"attempt_count,omitempty"`
EffectiveTestCommands []string
EquivalenceAI AIClient
EquivalencePrompt string
EquivalencePromptContext *PromptContext
EquivalencePromptMode EquivalencePromptMode
RecoveredAfterRetry bool `json:"recovered_after_retry,omitempty"`
func RenderEquivalencePromptForMode(promptCtx PromptContext, mode EquivalencePromptMode) (string, error)
RequireRepoContext bool
RetryCount int `json:"retry_count,omitempty"`
RetryReasons []string `json:"retry_reasons,omitempty"`
ReviewAI AIClient
ReviewPrompt string
ReviewPromptContext *PromptContext
```

The required exact string in the task is:

```text
could not be composed with the canonical reference test patch
```

For the displayed task, both models reached correctness rung 0.25 and failed the hidden tests. GPT 6.1 Sol scored 0.814 on the checklist, 0.952 on review, and 0.548 on production ship score; GPT 6 Sol scored 0.395, 0.762, and 0.405 respectively. GPT 6.1 Sol led the production ship score by 0.143, beyond the ±0.025 tie cutoff. The GPT 6.1 Sol patch changed 26 files, adding 1,487 lines and removing 89.

The checklist recorded 35 of 43 points. It marked these areas as met: focused diff, both replay paths, one shared replay policy, replaying only the test command, refusing module-wide Go targets, using recorded commands, fatal missing repository context, explicit equivalence prompt mode, recording attempts, using the original prompt for repairs, exact repair inputs, checking every repaired review, single-client compatibility, and focused helper policy. It marked bounded retries for recoverable failures, adapted diff-format enforcement, retry-case tests, and per-grader-client tests as missed.

The code-review scores were: reuse over reinvention 4; abstraction fit 4; naming and contracts 3.7; scope restraint 4; preservation of existing guards 3.3; documented surface 3.7; and test style fit 4. The review described the retry and replay work as thoughtfully structured and well tested, and identified an authority issue involving configured repair prompts replacing persisted measurement prompts.

## Where GPT 6.1 Sol lands on the leaderboard

GPT 6.1 Sol ranked second of 14 at **$1.10 per task**. The board score was 64.0, with a 95% interval of 53.9–73.8, correctness mean 0.61, and median time 11.0 minutes per task. GPT 6 Sol ranked seventh with a board score of 55.9, correctness mean 0.64, cost $3.01 per task, and median time 12.7 minutes.

| Rank | Setup | Board score | 95% interval | Correctness mean | Cost/task | Median time | Head-to-head vs GPT 6.1 Sol |
|---:|---|---:|---|---:|---:|---:|---|
| 1 | Opus 5 · Claude Code · medium | 67.0 | 56.7–76.6 | 0.54 | $6.78 | 25.0 min | GPT 6.1 Sol: 7 wins, 9 ties, 8 losses |
| 2 | GPT 6.1 Sol · Codex · medium | 64.0 | 53.9–73.8 | 0.61 | $1.10 | 11.0 min | — |
| 3 | Grok 4.7 High · Cursor · high | 62.6 | 51.4–73.1 | 0.61 | $5.76 | 18.5 min | 10 wins, 5 ties, 9 losses |
| 4 | SWE-2 Max · Devin · high | 62.0 | 52.7–71.1 | 0.58 | No recorded cost | 50.9 min | 7 wins, 8 ties, 9 losses |
| 5 | Pareto 26.9 · Claude Code · medium | 59.3 | 48.8–69.4 | 0.62 | $12.47 | 38.2 min | 13 wins, 4 ties, 7 losses |
| 6 | GPT 6 Luna · Codex · max | 58.8 | 49.2–67.5 | 0.49 | $0.37 | 54.6 min | 10 wins, 7 ties, 7 losses |
| 7 | GPT 6 Sol · Codex · medium | 55.9 | 48.4–63.5 | 0.64 | $3.01 | 12.7 min | 9 wins, 10 ties, 5 losses |
| 8 | Haiku 5.5 · Claude Code · max | 52.4 | 41.0–63.6 | 0.46 | $3.85 | 46.7 min | 10 wins, 9 ties, 5 losses |
| 9 | Haiku 5.5 · Claude Code · low | 46.8 | 38.3–55.3 | 0.42 | $0.25 | 4.7 min | 13 wins, 7 ties, 4 losses |
| 10 | DeepSeek V4.1 Flash · dsh · max | 46.5 | 36.7–56.1 | 0.46 | $0.49 over 23 of 24 tasks | 46.6 min | 12 wins, 8 ties, 4 losses |
| 11 | Opus 4.8 · Claude Code · medium | 41.0 | 30.7–51.5 | 0.47 | $7.05 | 17.3 min | 15 wins, 6 ties, 3 losses |
| 12 | Muse Spark 1.3 · Muse Code · high | 33.9 | 25.2–42.8 | 0.41 | $0.10 | 10.6 min | 17 wins, 5 ties, 2 losses |
| 13 | GLM 5.3 Flash · Codex · high | 28.4 | 19.8–38.4 | 0.38 | $0.41 | 17.7 min | 16 wins, 6 ties, 2 losses |
| 14 | GPT 6 Luna · Codex · low | 21.4 | 12.7–31.2 | 0.28 | $0.05 | 2.2 min | 17 wins, 5 ties, 2 losses |

Correctness is an average 0–1 rung, not a test pass rate. Costs cover coding attempts and exclude judging. Haiku max includes one estimate; DeepSeek has 23 recorded costs. The leaderboard scores each run against the entire field, so head-to-head ties can still produce different overall ranks.

## Diving deeper into signals

- **Correctness:** GPT 6.1 Sol scored lower on 2 tasks and the same on 22, averaging 0.606 versus 0.638. Each model was equivalent to the merged change on 10 tasks.
- **Checklist:** GPT 6.1 Sol averaged 0.579 versus 0.537 and missed 202 of 470 scored items, versus 232 of 470 for GPT 6 Sol.
- **Code review:** GPT 6.1 Sol averaged 0.965 versus 0.917.
- **Patches:** GPT 6.1 Sol was bigger on 20 of 24 tasks, with a median of 303 changed lines versus 197. It had more test lines on 20 of 24 tasks; tests were 36% of its changed lines versus 29% for GPT 6 Sol, while merged changes were 42%.

Correctness was identical on 22 tasks; GPT 6.1 Sol was higher on no correctness comparisons and GPT 6 Sol was higher on 2. For checklist, GPT 6.1 Sol was higher on 15, within ±0.025 on 4, and lower on 5. For code review, GPT 6.1 Sol was higher on 12, within ±0.025 on 9, and lower on 3. The ±0.0247338777 cutoff is a descriptive display bin, not a calibrated significance test.

The 24 task identifiers used in the patch-size comparison are: 3169aefa, 15439c21, 43493c52, 2450ca2d, 89dfbc27, 232b50ba, 6f84e978, b28f4fe7, 5276aa3f, f0caded9, 66762b4c, 5d766dde, e928166f, 5e075641, a8716b88, 30506f8e, 60a0e6c2, 342d0ac0, bbbbae09, 87814583, f8ae1177, 2826765e, d2ab0c25, and e41c2e2d. Changed lines include additions and removals. Test lines are files named like `_test.go`, `.test.`, `.spec.`, or `test_*.py`, and files under `testdata`, `__tests__`, and `fixtures`; documentation counts as implementation. Implementation footprint is excluded from the board score.

## Cost and speed

Average recorded usage and coding cost per task, excluding judging:

| Metric | GPT 6 Sol | GPT 6.1 Sol |
|---|---:|---:|
| Uncached input | 260K | 189K |
| Cached input | 10.84M | 5.03M |
| Output | 32K | 22K |
| Total tokens | 11.13M | 5.24M |
| Cached share of input | 98% | 96% |
| Cost of cached input | $2.17 | $0.50 |
| Total cost | $3.01 | $1.10 |

GPT 6.1 Sol was cheaper on 22 of 24 tasks. It used fewer tokens on 19 of 24 tasks—about one-third fewer on the typical task. At the recorded rates, cached input cost $0.10 per million tokens for GPT 6.1 Sol versus $0.20 for GPT 6 Sol; input and output prices were the same. At GPT 6 Sol’s rates, GPT 6.1 Sol’s usage would cost $1.60 per task, so most of the $1.91 difference came from using fewer tokens.

GPT 6.1 Sol’s median time was 11.0 minutes versus 12.7 minutes, and it finished first on 11 of 24 tasks; it was not meaningfully faster.

The source note records OpenAI standard-tier API pricing checked on **2026-10-09**: input $2 and output $10 per million tokens for both models; cached input $0.20 per million for GPT 6 Sol and $0.10 per million for GPT 6.1 Sol. The source named for pricing is `developers.openai.com/api/docs/pricing`.

## Conclusion

GPT 6.1 Sol maintained GPT 6 Sol’s quality while cutting cost roughly in half through lower token usage and cheaper cached input. In our codebase, it produced larger diffs that could take longer or require more tokens to review, although the additional code scored better on checklist and review. Teams can measure their own merged changes before switching models.

[Evaluate your coding workflow](https://www.stet.sh/private)

### Compare on the leaderboard

- [GPT 6.1 Sol medium vs GPT 6 Sol medium](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-gpt-6-sol-medium)
- [GPT 6.1 Sol medium vs Haiku 5.5 max](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-haiku-5-5-max)
- [GPT 6.1 Sol medium vs Haiku 5.5 low](https://www.stet.sh/leaderboard/compare/gpt-6-1-sol-medium-vs-haiku-5-5-low)

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fblog%2Fgpt-6-1-sol-vs-gpt-6-sol&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fblog%2Fgpt-6-1-sol-vs-gpt-6-sol). Opening a form does not execute a tool.