STET■

flux-pr-5575

Zod (TypeScript) · W2 · GPT-5.3 Codex

graphql-go-tools (Go)sqlparser-rs (Rust)Zod (TypeScript)
W2W1
|
GPT-5.3 CodexGPT-5.4GPT-5.1 Codex Mini
pass_with_warn

Tests passed. 2/2 commands passed. Strength: weak.

69.2% run pass rate
Tier 1
primary equivalencepassedneeds generated testsweak signal riskcommand source driftnon equivalentfail
pnpm build
gold passagent pass
npx vitest run packages/zod/src/v4/classic/tests/refine.test.ts packages/zod/src/v4/mini/tests/refine.test.ts -t "type\ guard\ narrows\ output\ type|non\-type\-guard\ refine\ does\ not\ narrow|type\ guard\ narrows\ output\ type|non\-type\-guard\ refine\ does\ not\ narrow"
gold passagent pass

Partial score: 2/2

Publishable: noWeak signal risk: yesCache: miss

Trajectory

unknown · partial order only

Canonical trajectory missing; showing coarse derived order only.

patch written
Patch captured
#1

Stet captured agent.patch for this trial.

agent.patch
validation
Tests passed
#2
validation
equivalence
Equivalence judgment
#3

non_equivalent

validation
code review
Code review judgment
#4

fail

validation
decision
Final decision
#5

pass_with_warn

validation

Quality

equivalence
non_equivalent
79% confidence
code review
fail
2 findings
footprint
low (0.28)
behavioral
100.0%
cost
$2.87 · 1.1M

Equivalence Reasoning

behavioral

The patch implements predicate-based narrowing for `classic` `.refine()` but appears to miss the corresponding `mini` schema method updates (`ZodMiniType` interface + `inst.refine` assignment). Gold behavior narrows for `.refine()` in both classic and mini APIs; this patch only updates mini’s standalone `refine` helper overload, so downstream `.refine()` typing/behavior parity is incomplete.

Code Review

correctness: 2/4introduced bug risk: 2/4edge case handling: 1/4maintainability idioms: 2/4

The patch moves toward predicate-aware refine typing but likely does not fully satisfy the task because mini schema method-level narrowing appears missing, and added `any` casts reduce confidence in robust type-safety.

2 findings
Mini `.refine()` type narrowing path appears incomplete
major

The patch adds predicate-aware method typing in classic `ZodType`, but mini changes only touch the standalone `refine` helper overload and cast return. Without matching mini method/interface wiring, callers of mini `.refine()` are unlikely to get the intended narrowed output type.

packages/zod/src/v4/classic/schemas.ts:88
Unsafe casts weaken the type-safety goal of the change
minor

New `as any` casts around refine method/helper implementations bypass type checking and can hide future typing regressions, which is counter to a type-narrowing feature.

packages/zod/src/v4/classic/schemas.ts:203
Evidencevalidation (97.8 KB)results (70.9 KB)run_metadata (1.6 KB)agent_patch (4.4 KB)summary (257.4 KB)manifest (695 B)