STET

flux-pr-5187

Zod (TypeScript) · W2 · GPT-5.1 Codex Mini

fail_high_conf

Tests failed. 0/1 commands passed. Strength: weak.

61.5% run pass rate
Tier 1
primary equivalencefailedneeds generated testsweak signal riskcommand source driftnon equivalentunsure
npx vitest run packages/zod/src/v4/core/tests/locales/es.test.ts -t "Spanish\ locale\ \-\ type\ name\ translations\ in\ too_small\ errors|Spanish\ locale\ \-\ type\ name\ translations\ in\ too_big\ errors|Spanish\ locale\ \-\ type\ name\ translations\ in\ invalid_type\ errors|Spanish\ locale\ \-\ fallback\ for\ unknown\ type\ names|Spanish\ locale\ \-\ other\ error\ cases"
gold passagent fail

Partial score: 0/1

Publishable: noWeak signal risk: yesCache: miss

Trajectory

unknown · partial order only

Canonical trajectory missing; showing coarse derived order only.

patch written
Patch captured
#1

Stet captured agent.patch for this trial.

validation
Tests failed
#2
equivalence
Equivalence judgment
#3

non_equivalent

code review
Code review judgment
#4

unsure

decision
Final decision
#5

fail_high_conf

Quality

equivalence
non_equivalent
86% confidence
code review
unsure
2 findings
footprint
low (0.15)
behavioral
0.0%
cost
$0.25 · 389K

Equivalence Reasoning

behavioral

The patch applies type-name translation in the right error branches, but the Spanish type map is incomplete (e.g., missing `enum`, `union`, `literal`, `void`, `never`, `unknown`, `any`, etc.), so some expected/received labels can still appear as raw English type names. It also changes translation choices (e.g., `string -> cadena`) that may diverge from the intended locale outputs.

Code Review

correctness: 2/4introduced bug risk: 2/4edge case handling: 2/4maintainability idioms: 3/4

The patch moves in the right direction and touches the required error codes, but it likely does not fully satisfy the intended change because type-name localization is not comprehensive and some translated terms appear mismatched to expected outputs.

2 findings
Type translation table is incomplete for known Zod type labels
major

The new `TypeNames` map omits labels like `enum`, `union`, `literal`, `void`, `never`, `unknown`, and `any`, so `invalid_type`/origin-based messages can still emit raw English for known types instead of consistent Spanish translations.

packages/zod/src/v4/locales/es.ts:13
Spanish terms differ from expected locale wording
major

The patch uses alternate wording (`string` -> `cadena`, `bigint` -> `entero grande`) rather than the expected locale vocabulary, which is likely to fail message-level tests tied to exact Spanish phrasing.

packages/zod/src/v4/locales/es.ts:14