Stet compares coding models on real merged-code tasks, then scores the patches above test pass rate: equivalence, code review, footprint risk, runtime, and cost.
56 tasks · Zod and graphql-go-tools
GPT-5.5 beat GPT-5.4 on tests, equivalence, review pass, and clean pass across 56 real coding tasks.
56 tasks · Zod and graphql-go-tools
GPT-5.5 was the better shipping default, while Opus 4.7 remained the lower-footprint option.
56 tasks · Zod and graphql-go-tools
GPT-5.4 matched the human patch more often than Opus 4.7, but Opus kept patches smaller and lower-risk.
28 tasks · Zod
Opus 4.7 did not pass more tests, but it produced tighter and more equivalent patches.
GPT-5.4 was stronger on equivalence in the 56-task benchmark: 35 of 56 patches matched the human change, versus 19 of 56 for Opus 4.7. Opus had the lower footprint risk, 0.20 versus 0.34, which makes it the more conservative patch writer.
| Metric | Opus 4.7 | GPT-5.4 |
|---|---|---|
| Tests pass | 33 / 56 | 31 / 56 |
| Equivalence | 19 / 56 | 35 / 56 |
| Clean pass | 10 / 56 | 11 / 56 |
| Footprint risk | 0.20 | 0.34 |