STET

flux-pr-5222

Zod (TypeScript) · W2 · GPT-5.1 Codex Mini

pass_with_warn

Tests passed. 1/1 commands passed. Strength: strong.

61.5% run pass rate
Tier 1
primary testspassednon equivalentfail
pnpm test
gold passagent pass

Partial score: 1/1

Publishable: yesCache: miss

Trajectory

unknown · partial order only

Canonical trajectory missing; showing coarse derived order only.

patch written
Patch captured
#1

Stet captured agent.patch for this trial.

validation
Tests passed
#2
equivalence
Equivalence judgment
#3

non_equivalent

code review
Code review judgment
#4

fail

decision
Final decision
#5

pass_with_warn

Quality

equivalence
non_equivalent
88% confidence
code review
fail
3 findings
footprint
low (0.30)
behavioral
100.0%
cost
$2.46 · 8.3M

Equivalence Reasoning

behavioral

The agent covers version bumps and async codec typing, but it misses key intended behavior for stub manifests: it does not add `"**/package.json"` to `packages/zod/package.json` `files`, so generated stubs may not be included in distributable artifacts. Its stub generator also targets `index.js` (and skips `src`) instead of directories with `index.d.cts`, which can miss required type-entry stub locations. Additionally, it enforces non-undefined defaults at runtime rather than tightening the default API typing (`NoUndefined`) as intended for type-system safety.

Code Review

correctness: 1/4introduced bug risk: 1/4edge case handling: 1/4maintainability idioms: 2/4

The agent patch partially addresses the task (version bump and async codec typing changes) but likely does not satisfy the intended build/scaffolding behavior and introduces additional runtime behavior changes with regression risk.

3 findings
Stub manifest generator targets wrong files and excludes key tree
major

The new script detects `index.js` and skips `src`, but the intended behavior is to generate stubs for directories containing `index.d.cts`. This can prevent required `package.json` stubs from being created where TypeScript resolution needs them.

scripts/write-stub-package-jsons.ts:15
New runtime throw for `prefault` may break existing valid schemas
major

The patch adds `assertDefined` checks for `prefault` values, causing runtime errors when a prefault resolves to `undefined`. This behavior change is broader than the requested non-undefined default tightening and risks compatibility regressions.

packages/zod/src/v4/classic/schemas.ts:1797
Codec async type update uses potentially inconsistent namespace
major

Async codec callbacks were changed to `util.MaybeAsync` in classic/mini schemas, while related core types use `core.util.MaybeAsync`. If `util.MaybeAsync` is not the same exported type in these modules, this can introduce typing inconsistencies or compile issues.

packages/zod/src/v4/classic/schemas.ts:1937