# Composer 2.5 was 6.5× cheaper on 50 real Rust and Go PRs

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/blog/composer-25-cheaper-where-its-safe
Source observed: 2026-11-15T21:02:54.792Z
Last checked: 2026-11-19T11:08:43.54+00:00
Indexed: 2026-11-15T21:07:26.121+00:00

---

# Composer 2.5 was 6.5× cheaper on 50 real Rust and Go PRs. Here’s where it’s actually safe.

**June 18, 2026 · 8 min read** · [Ben, Founder, Stet](https://www.stet.sh/about)

We compared Composer 2.5, Opus 4.8, GPT-5.5, and Opus 4.7 on **50 real merged PRs**: 25 from `sqlparser-rs` in Rust and 25 from `graphql-go-tools` in Go. Each task used a frozen repository snapshot, one attempt, and grading beyond pass/fail with [Stet](https://www.stet.sh/methodology). GPT-5.4 was the blinded judge.

Composer was **6.47× cheaper than Opus 4.8 on Rust** and about **7× cheaper on Go**, and took about **254 seconds versus 489 seconds** for Opus on the captured Rust arm. It tied GPT-5.5 on pooled tests at **44/50**, while Opus 4.8 reached **47/50**. Tests were near-saturated and did not separate quality. Opus 4.8 led Composer on craft in both languages, with the difference surviving multiplicity correction. Composer ranked last on craft in both repositories.

The useful boundary is task shape: Composer performed best on bounded, proportional, test-covered changes. Escalate parser, planner, public-API, defaults/null, golden-output, exact-source-text, and cross-package-contract work to a premium model.

## Cost versus local score

Our repo-balanced local display score uses:

**5% tests + 30% equivalence + 25% code review + 25% craft + 15% footprint**

Tests are downweighted because every arm cleared at least 84% on this slice. Cost is deliberately excluded from the quality score and shown as the second chart axis.

| Model | Repo-balanced local score | Cost per task |
|---|---:|---:|
| Opus 4.8 high | 74.4 | $3.72 |
| GPT-5.5 high | 70.0 | $4.05 |
| Opus 4.7 xhigh | 66.1 | $4.74 |
| Composer 2.5 | 64.6 | $0.62 |

Higher local score is better; lower cost is better. The interactive comparison can switch the quality axis among local score, craft, code review, equivalence, tests, and footprint.

### Per-axis comparison

| Axis | Opus 4.8 | GPT-5.5 | Opus 4.7 | Composer 2.5 |
|---|---:|---:|---:|---:|
| Craft, Rust | 3.28 | 2.94 | 2.98 | 2.84 |
| Craft, Go | 2.90 | 2.72 | 2.63 | 2.48 |
| Code review, Rust | 3.32 | 3.03 | 3.44 | 3.20 |
| Code review, Go | 2.76 | 2.51 | 2.29 | 2.25 |
| Equivalence, Rust | 0.92 | 0.88 | 0.72 | 0.80 |
| Equivalence, Go | 0.40 | 0.44 | 0.28 | 0.28 |
| Tests, pooled | 47/50 | 44/50 | 42/50 | 44/50 |
| Cost/task, Rust | $3.02 | $3.41 | $3.55 | $0.53 |
| Cost/task, Go | $4.41 | $4.69 | $5.93 | $0.71 |
| Footprint, pooled | 77.0 | 72.2 | 76.5 | 68.8 |

## Statistical strength

- **Opus 4.8 vs Composer 2.5:** Opus’s craft lead survived BH-FDR correction at `q=0.05` in both repositories: 7/11 quality-family graders in Rust and 10/11 in Go. Composer was 6.47× cheaper in Rust, decision-grade with CI excluding 1.0; the approximately 7× Go saving was recovered from logs and directional.
- **Opus 4.8 vs GPT-5.5:** Opus cleared BH-FDR on Rust craft for 4/11 quality-family graders; the Go craft gap was weak/directional. Opus won tokens decision-grade in both repositories; cost was decision-grade on Rust at 0.81× and in the noise band on Go at 0.83× after a cache-pricing fix.
- **Opus 4.8 vs Opus 4.7 xhigh:** Opus 4.8 was directionally ahead on craft and equivalence; Opus 4.7 had a Rust code-review-overall edge. Opus 4.8 was cheaper on Go; Rust was a statistical wash.

**Decision-grade (DG)** means surviving BH-FDR multiplicity correction at `q < 0.05`. **Directional** means consistent across graders without clearing that threshold. **Noise-band** means indistinguishable at this sample size.

## Two task classes, two outcomes

The same Composer 2.5 can be cheap and clean on a bounded tokenizer task, yet pass tests and fail review on a planner task.

### `graphql-go-tools` PR #1380: `@requires`/`@provides` resolver

The task was to resolve field dependencies in the planner. Tests passed, but review rejected the patch. Composer added a new **130-line ancestor-walk helper**, producing **+282 / −6 across 6 files** with craft **2.14**. The patch was non-equivalent because the test suite covered the happy path but did not assert that the candidate made the same behavioral change as the human PR.

On the same task, Opus 4.8 stayed behaviorally equivalent in roughly half the diff, although review flagged both patches.

### `graphql-go-tools` PR #1260

Composer produced **+272 / −26 across 8 files**, and the suite broke. Craft was **2.54**. We do not count this as a green-test win; the broader pattern is that local reasoning can look complete while leaving gaps the harness does not catch upstream.

Every Rust patch passed its suite, **25/25**. Go reached **19/25** tests, but the equivalence grader marked **15 of those 19** as not doing what the human did. A multi-megabyte build artifact even passed all tests; a cheap size guardrail caught it instead.

## Three buckets by task shape

### Replacement zone: bounded Rust changes

Composer was safe in this local slice when the task was bounded, proportional, and the tests exercised the actual obligation:

- `sqlparser-rs #1620`: SQLite dollar-placeholder tokenizer; equivalent, review-passing, craft **3.74**, **+54 / −2 across 4 files**.
- `sqlparser-rs #1472`: Hive `!` negation versus PostgreSQL `!` factorial disambiguation; equivalent and review-passing, craft **3.68**.
- `sqlparser-rs #1493`: `JSON_TABLE FOR ORDINALITY` with `NESTED PATH`; equivalent and review-passing, craft **3.53**. Review flagged a public-AST-shape maintainability issue, but the review gate held.

The fingerprint is bounded scope, tests covering the real obligation, proportional changes, and no contract drift.

### Composer-first, then grade: careful Go use

Some tasks were useful first drafts but not merge-ready on green CI:

- **#1001, HTTP observability:** tests passed and the patch was equivalent, but review failed because response metadata was omitted on body-read failure and request metadata was captured before redirects.
- **#859, planner:** tests passed and the patch was equivalent, but review found lazy metadata caches that could become stale without invalidation.

### Escalate immediately

Send these to Opus or GPT:

- Parser and planner behavior changes.
- Public API or type-shape changes.
- Checked-in expected or golden outputs.
- Exact source-text fidelity.
- Defaults and null behavior.
- Cross-package contracts.
- Changes spanning many files or layers.

Composer can pass tests and still produce code that review rejects or code that compiles while doing the wrong downstream thing.

## Route Composer by task shape

At dispatch time, use visible task characteristics: subsystem, additive versus structural obligation, and blast radius. Our routing principles are:

1. **Route by subsystem and obligation.** Composer fit bounded, additive grammar and tokenizer changes with one integration point. It struggled with planner, resolver, federation, invariants, and refactors such as preserving ordering or consolidating duplicated logic.
2. **Escalate on blast radius.** Public API/type changes, defaults/null behavior, checked-in outputs, cross-package contracts, and multi-layer changes go to a premium model regardless of test results.
3. **Gate output cheaply.** Size and diff-proportionality checks catch oversized patches without an oracle. A 40-line task returning 200 lines across three times as many files is a warning sign.
4. **Send green-but-bounded work to human review** when review continues to find edge defects; Composer is a fast first draft in this category.

The Rust-versus-Go split shows the replacement zone does not automatically transfer between repositories. Model updates and task mix also move the boundary, so we re-measure and let routing follow the evidence.

## What this is and is not

This is a local slice of **50 tasks across 2 open-source repositories**, run in late May 2026. The judge was GPT-5.4, blinded to model labels; each model used one seed and one attempt per task. It is not a general benchmark. Composite leaderboards place Composer last overall, but the actionable question is where it is safe for your code and where it leaks cost into review.

Composer is a cost lever inside the replacement zone and a liability outside it. Default to premium models for parser, planner, API-shape, defaults, and golden-output work; route only bounded, test-covered changes to Composer. [Replay your own merged PRs with Stet](https://www.stet.sh/private) before changing a routing default.

## FAQ

### Is Composer 2.5 cheaper than Opus 4.8?

Yes. It was 6.47× cheaper on Rust, decision-grade with CI excluding 1.0, and about 7× cheaper on Go, recovered from raw Cursor logs and directional. Rust duration favored Composer; Go duration was not captured.

### Can Composer 2.5 replace a premium model?

Only for selected task classes. In this slice it was safe on bounded, test-covered `sqlparser-rs` changes such as dialect additions and tokenizer tweaks. It was risky for parser, planner, and public-API work.

### Where is Composer safe to use?

Use it for bounded local changes where tests exercise the obligation and the patch remains proportional. Escalate parser and planner behavior, public API shape, and checked-in expected outputs.

### Did Composer beat premium models beyond cost?

It tied GPT-5.5 on pooled test pass rate at 44/50. It ranked last or near-last on craft, code review, and equivalence in both repositories.

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fblog%2Fcomposer-25-cheaper-where-its-safe&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fblog%2Fcomposer-25-cheaper-where-its-safe). Opening a form does not execute a tool.