Look at the data
Opus 4.8 vs Opus 5 — 25-task compare
Every matched task from the comparison, with both arms side by side. Expand a row to read the task instruction, the grader verdicts behind each score, the test commands that ran, and the per-file patch footprint.
baselineOpus 4.8 (claude-opus-4-8)
candidateOpus 5 (claude-opus-5)
source root.stet/studies/opus48-vs-opus5-medium-n25-20260803/requalify-provenance-first-live-75f1fe66-20260808-20260808T070031Z/compare
generated2026-08-20 20:54Z · 25 tasks
| task | tests | equivalence | footprint | cost | tokens | duration | patch +/- | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Opus 4.8 | Opus 5 | Opus 4.8 | Opus 5 | Opus 4.8 | Opus 5 | Opus 4.8 | Opus 5 | Opus 4.8 | Opus 5 | Opus 4.8 | Opus 5 | Opus 4.8 | Opus 5 | |
| pass | pass | equivalent | equivalent | 0.04 low | 0.02 low | $0.96 | $0.42 | 1.2M | 488K | 16.7m | 5.3m | +4/-5 | +0/-5 | |
| fail | fail | non equivalent | not recorded | 0.37 medium | 0.41 medium | $10.8 | $9.98 | 14.8M | 14.0M | 31.1m | 31.3m | +1052/-26 | +1808/-61 | |
| pass | pass | equivalent | equivalent | 0.32 low | 0.46 medium | $0.78 | $0.99 | 766K | 1.3M | 19.6m | 13.4m | +59/-1 | +51/-3 | |
| fail | pass | non equivalent | equivalent | 0.41 medium | 0.33 medium | $0.74 | $0.68 | 837K | 818K | 16.5m | 22.2m | +38/-3 | +44/-0 | |
| fail | fail | non equivalent | not recorded | 0.19 low | 0.49 medium | $14.6 | $15.7 | 20.4M | 22.5M | 63.8m | 30.6m | +1249/-19 | +1853/-33 | |
| fail | fail | non equivalent | not recorded | 0.35 medium | 0.55 medium | $15.3 | $14.3 | 24.0M | 22.7M | 38.4m | 43.8m | +722/-100 | +1072/-142 | |
| pass | pass | equivalent | equivalent | 0.33 medium | 0.32 low | $0.83 | $0.84 | 949K | 939K | 3.4m | 18.6m | +53/-1 | +46/-1 | |
| fail | fail | non equivalent | not recorded | 0.17 low | 0.34 medium | $9.67 | $4.90 | 14.3M | 6.9M | 61.1m | 16.9m | +384/-17 | +499/-64 | |
| pass | pass | equivalent | equivalent | 0.36 medium | 0.36 medium | $0.63 | $0.39 | 482K | 328K | 8.3m | 2.8m | +116/-1 | +102/-1 | |
| pass | pass | equivalent | equivalent | 0.31 low | 0.32 low | $0.95 | $0.85 | 928K | 875K | 7.5m | 9.7m | +83/-32 | +113/-26 | |
| fail | fail | equivalent | non equivalent | 0.32 low | 0.46 medium | $2.69 | $2.95 | 3.2M | 4.0M | 11.0m | 11.4m | +227/-51 | +415/-37 | |
| fail | fail | non equivalent | not recorded | 0.43 medium | 0.69 high | $10.6 | $9.94 | 15.0M | 14.5M | 29.8m | 31.5m | +846/-139 | +1242/-263 | |
| fail | fail | non equivalent | non equivalent | 0.19 low | 0.27 low | $10.7 | $6.88 | 15.5M | 9.9M | 22.7m | 25.2m | +394/-3 | +672/-21 | |
| fail | fail | equivalent | equivalent | 0.21 low | 0.35 medium | $1.63 | $1.42 | 1.9M | 1.4M | 27.3m | 37.2m | +120/-0 | +327/-0 | |
| pass | pass | equivalent | equivalent | 0.45 medium | 0.45 medium | $1.11 | $1.18 | 1.2M | 1.5M | 7.1m | 35.3m | +76/-2 | +78/-11 | |
| fail | fail | non equivalent | not recorded | 0.09 low | 0.45 medium | $5.90 | $13.8 | 7.4M | 21.4M | 16.8m | 33.7m | +458/-42 | +1149/-172 | |
| pass | pass | equivalent | equivalent | 0.01 low | 0.36 medium | $0.68 | $1.24 | 869K | 1.5M | 16.8m | 33.2m | +8/-1 | +152/-1 | |
| fail | fail | non equivalent | not recorded | 0.39 medium | 0.52 medium | $21.5 | $11.4 | 34.8M | 17.4M | 38.6m | 43.5m | +536/-85 | +957/-84 | |
| pass | pass | equivalent | equivalent | 0.01 low | 0.01 low | $1.01 | $0.54 | 1.5M | 641K | 19.1m | 20.7m | +18/-10 | +8/-2 | |
| pass | fail | equivalent | not recorded | 0.20 low | 0.30 low | $9.99 | $7.38 | 12.8M | 9.7M | 29.5m | 19.1m | +1302/-4 | +1583/-5 | |
| fail | fail | non equivalent | not recorded | 0.14 low | 0.32 low | $5.87 | $14.1 | 7.7M | 22.7M | 13.7m | 26.3m | +532/-12 | +1074/-70 | |
| fail | fail | non equivalent | non equivalent | 0.06 low | 0.22 low | $4.01 | $11.7 | 4.8M | 18.3M | 14.2m | 38.7m | +105/-31 | +588/-47 | |
| fail | fail | non equivalent | non equivalent | 0.27 low | 0.60 medium | $2.26 | $5.32 | 1.9M | 7.7M | 25.3m | 25.6m | +339/-0 | +450/-1 | |
| fail | fail | equivalent | equivalent | 0.17 low | 0.25 low | $3.78 | $2.95 | 4.7M | 3.8M | 40.5m | 11.3m | +156/-65 | +253/-50 | |
| fail | fail | not recorded | not recorded | 0.19 low | 0.36 medium | $24.3 | $17.4 | 37.5M | 26.4M | 46.5m | 28.0m | +1418/-11 | +1921/-18 | |