STET

flux-pr-4539

Zod (TypeScript) · W2 · GPT-5.4

pass_with_warn

Tests passed. 3/3 commands passed. Strength: weak.

69.2% run pass rate
Tier 1
primary equivalencepassednon equivalentdecision conflictfailneeds generated testsweak signal riskcommand source drift
pnpm build
gold passagent pass
find . -name vitest.config.ts -exec sed -i 's/test: {/test: { testTimeout: 30000,/' {} +
gold passagent pass
npx vitest run packages/zod/src/v4/classic/tests/default.test.ts -t "nested\ prefault/default|failing\ default"
gold passagent pass

Partial score: 3/3

Publishable: noWeak signal risk: yesCache: miss

Trajectory

unknown · partial order only

Canonical trajectory missing; showing coarse derived order only.

patch written
Patch captured
#1

Stet captured agent.patch for this trial.

validation
Tests passed
#2
equivalence
Equivalence judgment
#3

non_equivalent

code review
Code review judgment
#4

fail

decision
Final decision
#5

pass_with_warn

Quality

equivalence
non_equivalent
99% confidence
code review
fail
2 findings
footprint
high (1.00)
behavioral
100.0%
cost
$0.65 · 1.1M

Equivalence Reasoning

behavioral

The intended change is to fix object optionality detection by considering both `optin` and `optout` in schema/object validation paths. The agent patch (as shown) only adds `app/node_modules/.bin/*` wrapper scripts and does not modify the relevant Zod core logic (`packages/zod/src/v4/core/schemas.ts`, `packages/zod/src/v4/core/util.ts`) or any equivalent behavior. This misses the core functional requirement.

Code Review

correctness: 0/4introduced bug risk: 0/4edge case handling: 0/4maintainability idioms: 0/4

This agent patch is almost certainly incorrect for the task: it adds generated node_modules launcher scripts instead of implementing the required Zod optionality/default/prefault logic changes.

2 findings
Patch does not implement requested schema logic changes
major

The intended fix requires updates in Zod object/schema optionality handling, but this patch only adds node_modules .bin wrapper scripts and therefore does not implement the described behavior change.

app/node_modules/.bin/attw:1
Generated binaries with hardcoded /app paths add fragility
major

New wrappers embed absolute /app dependency paths in NODE_PATH and executable targets, which is brittle outside the specific build environment and can break local/CI behavior.

app/node_modules/.bin/attw:8