STET

flux-pr-3850

Zod (TypeScript) · W2 · GPT-5.4

fail_high_conf

Tests failed. 3/4 commands passed. Strength: strong.

69.2% run pass rate
Tier 1
primary testsfailednon equivalentcommand source drift
yarn build
gold passagent pass
find . -name vitest.config.ts -exec sed -i 's/test: {/test: { testTimeout: 30000,/' {} +
gold passagent pass
yarn test
gold passagent pass
pytest -q tests/behavior/standard_schema_validate_behavior_test.py
gold passagent fail

Partial score: 3/4

Publishable: yesCache: miss

Trajectory

unknown · partial order only

Canonical trajectory missing; showing coarse derived order only.

patch written
Patch captured
#1

Stet captured agent.patch for this trial.

validation
Tests failed
#2
equivalence
Equivalence judgment
#3

non_equivalent

code review
Code review judgment
#4

fail

decision
Final decision
#5

fail_high_conf

Quality

equivalence
non_equivalent
88% confidence
code review
fail · 14/100
4 findings
footprint
high (1.00)
behavioral
75.0%
cost
$1.07 · 1.6M

Equivalence Reasoning

behavioral

The patch adds a `standard-schema` type file and tests, but it does not clearly implement the core runtime integration in schema types (the `~standard` metadata + `validate` hook with sync-to-async fallback) shown by the intended change. From the diff shown, `deno/lib/types.ts` only adds an import, so the main behavior appears missing; the patch also includes many unrelated generated files.

Code Review

correctness: 1/4introduced bug risk: 0/4edge case handling: 1/4maintainability idioms: 0/4

The agent patch likely does not satisfy the intended Standard Schema runtime integration and introduces substantial unrelated artifact churn, so it is high-risk despite passing tests.

4 findings
Runtime Standard Schema hook is not implemented on core type
major

The patch only adds an import of `StandardSchemaV1` in `types.ts` without adding `~standard` initialization or validation bridging on the base schema class, so consumers likely cannot rely on the runtime standard interface.

app/deno/lib/types.ts:24
Standard Schema type surface is incompatible with expected spec shape
major

The custom `standard-schema.ts` omits spec-like optional type metadata and path segment shape and adds a non-standard `async` property, reducing compatibility with external Standard Schema tooling.

app/deno/lib/standard-schema.ts:6
Patch includes large unrelated built artifacts
major

New compiled files under `app/lib` (runtime JS, d.ts, tests, benchmarks) are unrelated to the requested behavior and materially increase risk of accidental regressions and packaging noise.

app/lib/ZodError.js:1
Public export added without corresponding end-to-end implementation
minor

Exporting `standard-schema.ts` from `external.ts` publishes a new API surface while the core runtime integration appears incomplete, creating a misleading contract for maintainers and consumers.

app/deno/lib/external.ts:5