STET

flux-pr-4811

Zod (TypeScript) · W2 · GPT-5.1 Codex Mini

pass_with_warn

Tests passed. 3/3 commands passed. Strength: weak.

61.5% run pass rate
Tier 1
primary equivalencepassedneeds generated testsweak signal riskcommand source driftnon equivalentfail
pnpm build
gold passagent pass
find . -name vitest.config.ts -exec sed -i 's/test: {/test: { testTimeout: 30000,/' {} +
gold passagent pass
npx vitest run packages/zod/src/v4/classic/tests/to-json-schema.test.ts -t "number\ constraints\ draft\-4|literal\ draft\-4"
gold passagent pass

Partial score: 3/3

Publishable: noWeak signal risk: yesCache: miss

Trajectory

unknown · partial order only

Canonical trajectory missing; showing coarse derived order only.

patch written
Patch captured
#1

Stet captured agent.patch for this trial.

validation
Tests passed
#2
equivalence
Equivalence judgment
#3

non_equivalent

code review
Code review judgment
#4

fail

decision
Final decision
#5

pass_with_warn

Quality

equivalence
non_equivalent
99% confidence
code review
fail
3 findings
footprint
high (1.00)
behavioral
100.0%
cost
$1.64 · 4.0M

Equivalence Reasoning

behavioral

The agent patch does not implement the requested JSON Schema draft-04 support (no changes to JSON Schema generator logic, target type/options, `$schema` URI handling, or docs). It appears to add generated `node_modules/.bin/*` scripts, which are unrelated to the task’s intended behavior.

Code Review

correctness: 0/4introduced bug risk: 1/4edge case handling: 0/4maintainability idioms: 0/4

The agent patch likely does not satisfy the intended draft-4 JSON Schema change; it appears to add unrelated node_modules binary wrappers instead of implementing the required documentation and generator updates.

3 findings
Patch does not implement requested draft-4 JSON Schema feature
major

The submitted diff content is composed of node_modules .bin wrapper additions rather than updates to JSON Schema docs/generator files required by the task, so the intended behavior change is missing.

app/node_modules/.bin/attw:1
Generated dependency artifacts are committed
major

Committing many generated executables under node_modules is high-noise, non-source output that is typically excluded from reviewable feature patches and harms maintainability.

app/node_modules/.bin/prettier:1
Hardcoded absolute /app paths in scripts
major

The added wrappers export NODE_PATH values rooted at /app, which can fail outside that filesystem layout and create environment-specific breakage.

app/node_modules/.bin/tsc:13