Sonnet 5 vs Opus 4.8: how they behave and when to use each

Sonnet 5 is a confusing model. 5 > 4.8, but Opus > Sonnet. When should we use Sonnet 5? In this post, we used a private eval to test Sonnet 5 vs Opus 4.8 across 24 tasks from two open source repos and inspected the behavioral differences to answer when to reach for each model.

TL;DR: on these tasks, Sonnet turned higher reasoning effort into more checking, longer trajectories, and patches the LLM judge scored clearer and more intentional. Opus's activity stayed flatter through high reasoning effort, while the judge leaned toward simpler, more robust, more minimal diffs. Neither profile is universally "better", but the failure modes and working styles shift.

The kicker is price, and output tokens, which changes dramatically with reasoning effort. Sonnet cost 0.62x Opus at low and 0.81x at medium; high and xhigh were effectively tied; at max Sonnet cost 1.37x Opus.

The short answer

This is a routing policy from one local slice, a demonstration of the power of running evals on your own code, not a universal model hierarchy like DeepSWE or FrontierCode.

How we ran it

24 real tasks from two open-source repos: graphql-go-tools (Go) and sqlparser-rs (Rust), each derived from a PR that the maintainers actually merged. Both Sonnet and Opus ran every task at five reasoning efforts (low, medium, high, xhigh, max) in Claude Code, with one run per arm. One GPT-5.4 pointwise judge graded equivalence to the merged fix and eight craft dimensions. Costs are cache-aware geometric means per task.

The cost ladder

Sonnet's dial spans 9.0x, from $0.98 geomean per task at low to $8.77 at max; Opus's dial is flatter, about 4x from $1.58 to $6.41.

The quality trade

One pointwise judge scored eight craft dimensions on every patch pair: clarity, simplicity, coherence, intentionality, robustness, instruction adherence, scope discipline, and diff minimality.

Most individual margins are small, and the eight dimensions are not independent. I read this as a behavioral fingerprint, not a leaderboard. The signal is that the same judge saw the same directional trade across nearly every effort setting. Sonnet's patches were more often judged clear and intentional; Opus's were more often judged simple, robust, and minimal. We can use this to learn more about how the models performed on these tasks, but not to claim that either model writes better code in general.

Here's what leans consistently, and what it means:

Clarity leans Sonnet at all six comparison points. Strongest signal: high effort Sonnet takes 20 of 24 tasks with 2 ties; at xhigh 18 of 21. If reviewer comprehension is the bottleneck, this is the clearest case for Sonnet, especially at high and xhigh.

Intentionality leans Sonnet at all six points. That suggests Sonnet's changes may be easier for a reviewer to connect to the task.

Diff minimality leans Opus at all six points. Strongest: low effort Opus takes 21 of 24 tasks with zero ties; Sonnet xhigh versus Opus high is 17 of 22. This is the case for Opus when patch surface is policy, and it lines up with the deterministic footprint read.

Simplicity. The graders lean Opus on simplicity at every effort level except xhigh, where it tilts Sonnet's way, 11 tasks to 9. The judge more often read Opus as the less complicated fix; xhigh is where Sonnet closes that gap.

Robustness leans Opus at five of six. Again xhigh is the exception, and there it's effectively even at 9 to 11. When robustness is the bottleneck, the judge generally preferred Opus, while xhigh makes the models hard to separate.

Sonnet's share of the eight-dimension panel tracks the dial. Sonnet's share of the eight dimensions climbs as effort rises: 2 of 8 at low, 3 at medium, 4 at high, 7 of 8 at xhigh, then falls back to 4 at max. xhigh is the only setting where Sonnet carries most of the panel. That is why Sonnet xhigh, not low, high, or max, is the premium Sonnet route.

The tradeoff here seems to be either smaller, targeted patches (Opus) or easier to review code (Sonnet).

Footprint risk: what the patch actually touched

For every task, we compares the model's actual patch with the patch from the merged PR. The 0–100 score blends divergence from that reference with the patch's absolute change surface; lower means less measured change-surface risk.

Through high, the models stay close. The footprint expands at the top of the dial: Sonnet reaches 35.8 at max, while Opus reaches 30.8 at xhigh and 29.8 at max. On the 22 tasks shared by the two routes I recommend, Opus high averaged 21.5 versus Sonnet xhigh's 27.6 and had the lower score on 18 tasks. That supports Opus high when keeping the review surface narrow is part of the requirement; Sonnet xhigh can still be worth the larger surface when deeper verification matters more.

The lower panel answers how much of the added surface came from tests. At max, tests and fixtures make up 58.9% of Sonnet's added lines and 54.1% of Opus's. Across each route arm's included tasks, the mix is nearly identical, 46.8% for Sonnet xhigh and 47.3% for Opus high, so Sonnet xhigh's higher footprint risk is not simply “more tests.” Total churn, file breadth, non-test scope, and overlap with the merged patch also matter.

Smaller isn't always better though. A small patch can simply do too little, and a valid alternative architecture can look riskier than the maintainer's patch.

Matching the real fix

Additionally, we use the code the repository's maintainers actually merged as the answer key, graded on a 0-to-4 scale.

Opus barely moves with its reasoning effort. Mean scores across low through max: 3.00, 3.09, 3.16, 3.17, 3.14. Wherever you set Opus's dial, it lands about the same distance from the merged fix.

Sonnet climbs: 2.62 at low, 2.94 at medium, 2.93 at high, then 3.21 at xhigh, the only setting where it scores above Opus, before falling back to 2.91 at max. With Sonnet, more spend bought more equivalence through xhigh, then regressed at max.

Looking at the head to head comparisons: Opus leads at low (13 tasks to 6) and medium (11 to 4); they tie at high (8-8-8); Sonnet leads at xhigh (9 to 5); Opus comes back at max (10 to 7). For Sonnet xhigh versus Opus high: Sonnet 8, Opus 4, ties 10.

Sonnet's count of fully-equivalent patches rises with effort: 6 at low, 9 at medium and high of 24 tasks, and 10 of 22 at xhigh, before falling back to 8 of 24 at max.

How the behavior diverges

On this slice, Sonnet's measured activity rose as reasoning effort increased. Median session steps: 81 at low, climbing through 115.5, 156.5, 181, to 269 at max. Median output tokens climb the same ladder, from 13.7k at low to 113.6k at max. The first file edit lands at tool call 13 at low and tool call 55 at max: a longer runway before touching anything. Git commands per session climb every tier: 1, 2, 4, 6.5, 9.

The rest of the toolbox moves too. Median test commands per session go 2.5 → 5.5 → 7.5 → 7 → 9.5 low through max, grep and search commands 9 → 20 → 29 → 25 → 37, files touched from 3.5 at low to 8 at max, and patch churn (bytes rewritten vs bytes surviving) peaks at xhigh at about 2.3x.

Opus's measured activity stayed comparatively flat through high, then jumped. Session steps 78 → 100.5 → 98.5 through high, then 167 at xhigh and 261.5 at max. Test commands sit between 3.5 and 4.5 through high, then jump to 9 at xhigh. Its output tokens start above Sonnet's at low (20.3k) but stay near 34k through medium and high before climbing to 91.2k at max.

Quality follows Sonnet's scaling to a point. The graders score Sonnet's xhigh output above low and medium on all eight quality dimensions, and above high on six of the eight. The charts below carry the full set of counters across every effort level.

Where Sonnet's surplus effort goes matters. Two sessions illustrate it:

sqlparser#1472, Sonnet xhigh: same 10-edit, six-file footprint as the low run, shell commands 25 up to 79 (roughly 3x). The patch barely grew, but the number of commands run tripled.

graphql#1099, Sonnet xhigh: wrote a temporary repro test, cited GraphQL spec sections 5.6 and 5.8.5, confirmed the failure, reverted the test with git checkout, then implemented the real fix. The extra reasoning went into understanding, not additional changes.

Opus's sessions also grow much longer at xhigh and max.

Same task across the two models

The aggregate counters show the reasoning effort dials work differently for each model; watching both models on the identical task illustrates how. These are anecdotal sessions, not statistics, but they help illustrate the numbers above.

sqlparser#1398, both at low: same fix, different constitution. Both models converged on the same mechanism: a new trait method named require_interval_qualifier and a rejection site in the interval parser. The divergence emerged when the task spec collided with a pre-existing BigQuery test. Opus held the spec literally: kept BigQuery in the reject set, rewrote the conflicting test, and surfaced the conflict out loud: "If you specifically need BigQuery to accept that ISO-style bare-string form… let me know and I can reconsider BigQuery's classification." Sonnet treated the existing test suite as the authority: enforced the spec first, watched the full suite fail, then silently reverted BigQuery's enforcement mid-session with no narration at the moment of the revert. The rationale appears only in its final summary. One obeyed the spec and flagged the conflict; the other obeyed the tests and mentioned it after the fact.

sqlparser#1472, the dial moves different things. All four cells examined converged on the same mechanism: two new dialect-trait flags. What extra effort changed differed by model. For Opus, more effort changed WHAT got edited. At medium, it was the only cell to touch the permissive catch-all dialect file (generic.rs), flipping both flags on. At high, it checked the existing tests before editing, left that file alone, and called the narrower behavior "a deliberate behavior change." For Sonnet, more effort changed HOW MUCH got CHECKED, not what got touched. Low quoted the spec to itself: "Since spec says 'dialects that support neither should reject both usages'. Generic should reject." At xhigh it made the same call, then spent its extra budget on empirical verification ("Let's just empirically test it rather than guess."), clippy triage, and two extra full-suite runs. Where extra Opus effort altered what got edited, extra Sonnet effort altered how much got checked.

graphql#1128, both at xhigh: one question answered, then another. Both found the same planner bug and both wrote a throwaway repro test before editing. Both shipped near-identical planner fixes. The divergence is which question each considered answered. Sonnet's experiment lived entirely at the plan layer: it proved the planner failed, fixed it, confirmed the fixed planner emits no fetch, and stopped. It never tested what renders at runtime once the fetch is gone. Opus treated the passing plan tests as the start of a second question: "The root object has no TypeName, and the field reads __typename from empty data. Runtime would error. Let me confirm by writing a quick execution test." Its throwaway execution test demonstrated the runtime failure before it wrote the fallback fix in the resolve layer, then verified with the same test, deleted the scaffold, and converted the experiment into a permanent regression test covering all three operation types. Opus did all this in fewer tool calls (97 vs 129) while editing more of the system (8 files vs 5). At high effort Sonnet closed the same runtime gap by a third route, baking the value in at plan time, so this is one session's blind spot, not a fixed trait of the model.

Max: not worth the money

The evidence does not support max as a default for either model. Some examples here show why the extra spend needs a specific reason:

graphql#1076, Sonnet max: rewrote the same resolver event loop as the reference solution but touched three extra subscription-layer files, added three brand-new concurrency test files absent from the reference, then burned budget on repeated -race -count=8 stress loops.

graphql#1099, Sonnet max: repeated the same grep three times, re-read the same files. The final patch came out slightly smaller than xhigh's despite at least a quarter more logged commands. More grinding shell commands but a smaller overall patch.

sqlparser#2174, both at max: same design, different budgeting. Both independently converged on the same design the reference solution didn't use: an attribute macro backed by a table of all 142 dialect-method signatures. The difference was how each spent the max budget. Opus spent it reasoning out loud: a written design decision, lifetime analysis of the two reference-returning methods, and a drift test guarding the table. Sonnet spent it building tooling. 71 of its 117 tool calls were shell commands, and its signature move was writing a throwaway Rust program in /tmp to parse the trait and generate the table before touching the repo. It narrated almost nothing (16 assistant messages the whole session). Opus at high on the same task was the design outlier: it eliminated the signature table entirely by capturing the trait's methods at compile time, in 45 tool calls and under half the output tokens of either max run.

Opus at max carries one real signal. The clarity grader scores Opus max above Opus xhigh on 20 of 23 tasks with zero losses, the strongest within model result we have. But it costs 1.34x the xhigh price and it's one grader's lens.

I can't construct a routing rule that defaults to max for either model. The cost is clear; what you reliably get for it is too narrow for everyday use. Reserve max for when you know you need it, and you're confident the extra reasoning effort will help.

The routing rule

There is no single winner here, but we can extract a useful default. For the best cost/performance balance on these tasks, I would start with Opus high: its measured equivalence to the human patch was already near the top of the Opus curve, while cost rose sharply beyond high. I would route to Sonnet xhigh when the task is ambiguous, verification-heavy, or unusually expensive for a reviewer to understand; that is where Sonnet's equivalence, craft profile, and observed checking behavior line up most favorably. For routine work where high is unnecessary, Opus medium is the lower-effort floor I would use.

I would not default either model to max: Sonnet's measured quality curves fell back from xhigh, while Opus max's clarity signal was too narrow to justify the extra cost as a general rule.

This is 24 tasks on two repos with our graders. These leans and flips are properties of the task selection; the only numbers that should truly inform your routing decisions are ones from your own repos, with your own review patterns, and your own harness. Measure your own harness on your own code.