Stet compares coding models on real merged-code tasks, then scores the patches above test pass rate: equivalence, code review, footprint risk, runtime, and cost.
Public comparisons show how model and harness choices behaved on these repos. A private Stet run answers the same question with your PR history, tests, AGENTS.md, and review standards. Compare Claude Code vs Codex on your repo.
56 tasks · Zod and graphql-go-tools
GPT-5.5 beat GPT-5.4 on tests, equivalence, review pass, and clean pass across 56 real coding tasks.
56 tasks · Zod and graphql-go-tools
GPT-5.5 was the better shipping default, while Opus 4.7 remained the lower-footprint option.
56 tasks · Zod and graphql-go-tools
GPT-5.4 matched the human patch more often than Opus 4.7, but Opus kept patches smaller and lower-risk.
28 tasks · Zod
Opus 4.7 did not pass more tests, but it produced tighter and more equivalent patches.
26 tasks · graphql-go-tools
GPT-5.5 Codex at four reasoning settings on 26 matched graphql-go-tools tasks: equivalence and review pass climbed sharply with reasoning, while tests were not monotonic.
29 tasks · graphql-go-tools
Claude Opus 4.7 peaked at medium reasoning effort on 29 matched graphql-go-tools tasks; high, xhigh, and max were more expensive without improving the primary rollout metrics.
GPT-5.4 was stronger on equivalence in the 56-task benchmark: 35 of 56 patches matched the human change, versus 19 of 56 for Opus 4.7. Opus had the lower footprint risk, 0.20 versus 0.34, which makes it the more conservative patch writer.
| Metric | Opus 4.7 | GPT-5.4 |
|---|---|---|
| Tests pass | 33 / 56 | 31 / 56 |
| Equivalence | 19 / 56 | 35 / 56 |
| Clean pass | 10 / 56 | 11 / 56 |
| Footprint risk | 0.20 | 0.34 |