STET

flux-commit-7af773c0

Zod (TypeScript) · W2 · GPT-5.3 Codex

fail_high_conf

Tests failed. 1/6 commands passed. Strength: strong.

69.2% run pass rate
Tier 1
primary testsfailedcommand source driftnon equivalentfail
python3 -c "import json; c=json.load(open('jestconfig.json')); c['globals']={'ts-jest':{'diagnostics':False}}; json.dump(c,open('jestconfig.json','w'),indent=2)"
gold passagent pass
npx jest --config jestconfig.json --testPathPattern 'codegen|function|refine'
gold failagent
pytest -q tests/behavior/test_codegen_api_surface_behavior.py
gold passagent fail
pytest -q tests/behavior/test_codegen_optional_object_behavior.py
gold passagent fail
pytest -q tests/behavior/test_userpost_schema_behavior.py
gold passagent fail
pytest -q tests/behavior/test_index_and_tozod_behavior.py
gold passagent fail

Partial score: 1/5

Publishable: yesCache: miss

Trajectory

unknown · partial order only

Canonical trajectory missing; showing coarse derived order only.

patch written
Patch captured
#1

Stet captured agent.patch for this trial.

validation
Tests failed
#2
equivalence
Equivalence judgment
#3

non_equivalent

code review
Code review judgment
#4

fail

decision
Final decision
#5

fail_high_conf

Quality

equivalence
non_equivalent
78% confidence
code review
fail
3 findings
footprint
low (0.23)
behavioral
0.0%
cost
$4.75 · 1.6M

Equivalence Reasoning

behavioral

The patch is close, but it diverges in behavior that matters to the intended feature set. In `codegen.ts`, `isOptional` handles intersections with `||` instead of the expected `&&`, which can incorrectly mark required fields as optional in generated TS types. Also, the recursive typing example in `userpost.ts` was changed from `toZod<...>` to `ZodType<...>`, weakening the intended `toZod` recursive-schema typing path the task calls out. These are core intent-level differences, not just style.

Code Review

correctness: 2/4introduced bug risk: 2/4edge case handling: 1/4maintainability idioms: 2/4

Patch is close and likely passes targeted tests, but it diverges from intended semantics in intersection optionality and weakens robustness by silently defaulting unknown schema kinds to `any`; recursive sample typing/export shape also drifts from the intended change.

3 findings
Intersection optionality is computed with OR instead of AND
major

In `isOptional`, intersection schemas return optional when either side is optional. For `A & B`, undefined is valid only if both sides accept undefined, so this mislabels required object keys as optional in generated TS.

src/codegen.ts:20
Recursive sample schema no longer exposes intended toZod-typed exports
major

`userpost.ts` switched from exported `toZod`-typed schemas to non-exported `ZodType` variables, which weakens the intended public example/typing path for recursive schema typing flows.

src/userpost.ts:15
Unsupported schema kinds are silently emitted as any
major

The codegen default branch returns `any` rather than failing fast, which can hide missing cases and generate incorrect declarations without signaling an implementation gap.

src/codegen.ts:139