STET

flux-pr-4539

Zod (TypeScript) · W2 · GPT-5.1 Codex Mini

pass_with_warn

Tests passed. 3/3 commands passed. Strength: weak.

61.5% run pass rate
Tier 1
primary equivalencepassedneeds generated testsweak signal riskcommand source driftnon equivalentfail
pnpm build
gold passagent pass
find . -name vitest.config.ts -exec sed -i 's/test: {/test: { testTimeout: 30000,/' {} +
gold passagent pass
npx vitest run packages/zod/src/v4/classic/tests/default.test.ts -t "nested\ prefault/default|failing\ default"
gold passagent pass

Partial score: 3/3

Publishable: noWeak signal risk: yesCache: miss

Trajectory

unknown · partial order only

Canonical trajectory missing; showing coarse derived order only.

patch written
Patch captured
#1

Stet captured agent.patch for this trial.

validation
Tests passed
#2
equivalence
Equivalence judgment
#3

non_equivalent

code review
Code review judgment
#4

fail

decision
Final decision
#5

pass_with_warn

Quality

equivalence
non_equivalent
96% confidence
code review
fail
3 findings
footprint
high (1.00)
behavioral
100.0%
cost
$0.67 · 1.3M

Equivalence Reasoning

behavioral

The intended fix is to change optional-field detection logic (using both `optin` and `optout`) in object schema evaluation and optional-key reporting (e.g., `schemas.ts` and `util.ts`). The provided agent patch appears to add `node_modules/.bin/*` wrapper scripts and does not show changes to the relevant schema/optional-key logic, so it misses the core behavior.

Code Review

correctness: 0/4introduced bug risk: 0/4edge case handling: 0/4maintainability idioms: 0/4

The agent patch is very likely incorrect for the requested task: it primarily commits generated node_modules launcher scripts and does not show the targeted schema optionality/default handling changes.

3 findings
Patch appears off-target and does not implement schema optionality fix
major

The submitted diff adds node_modules .bin wrappers rather than changing zod object/schema logic, so it likely does not implement the required optin+optout optional detection behavior.

node_modules/.bin/attw:1
Hardcoded absolute /app paths in committed launchers
major

The new scripts export NODE_PATH entries rooted at /app, which is environment-specific and can fail in normal repository checkouts or CI variants.

node_modules/.bin/attw:8
Generated dependency artifacts were committed
major

Multiple generated executables under node_modules/.bin are included in the patch, creating noisy, non-source changes and poor maintainability.

node_modules/.bin/biome:1