STET

flux-pr-1169

graphql-go-tools (Go) · W2 · GPT-5.1 Codex Mini

pass_with_warn

Tests passed. 1/1 commands passed. Strength: strong.

96.6% run pass rate
Tier 1
primary testspassednon equivalentfail
go test -C v2 ./... -count=1 -timeout=300s
gold passagent pass

Partial score: 1/1

Publishable: yesCache: miss

Trajectory

codex · partial order only

provider-native trajectory captured; validation and decision steps are appended with coarse ordering only

session start
Session started
#1
tool call
Command started
#2

shell command

tool result
Command finished
#3

shell command exit code 0

tool call
Command started
#4

shell command

tool result
Command finished
#5

shell command exit code 0

tool call
Command started
#6

shell command

tool result
Command finished
#7

shell command exit code 0

tool call
Command started
#8

shell command

tool result
Command finished
#9

shell command exit code 0

tool call
Command started
#10

shell command

tool result
Command finished
#11

shell command exit code 0

tool call
Command started
#12

shell command

tool result
Command finished
#13

shell command exit code 0

tool call
Command started
#14

shell command

tool result
Command finished
#15

shell command exit code 0

tool call
Command started
#16

shell command

tool result
Command finished
#17

shell command exit code 0

tool call
Command started
#18

shell command

tool result
Command finished
#19

shell command exit code 0

tool call
Command started
#20

shell command

tool result
Command finished
#21

shell command exit code 0

tool call
Command started
#22

shell command

tool result
Command finished
#23

shell command exit code 0

tool call
Command started
#24

shell command

tool result
Command finished
#25

shell command exit code 0

tool call
Command started
#26

shell command

tool result
Command finished
#27

shell command exit code 0

tool call
Command started
#28

shell command

tool result
Command finished
#29

shell command exit code 0

tool call
Command started
#30

shell command

tool result
Command finished
#31

shell command exit code 0

tool call
Command started
#32

shell command

tool result
Command finished
#33

shell command exit code 0

tool call
Command started
#34

shell command

tool result
Command finished
#35

shell command exit code 0

tool call
Command started
#36

shell command

tool result
Command finished
#37

shell command exit code 0

tool call
Command started
#38

shell command

tool result
Command finished
#39

shell command exit code 0

tool call
Command started
#40

shell command

tool result
Command finished
#41

shell command exit code 0

tool call
Command started
#42

shell command

tool result
Command finished
#43

shell command exit code 0

tool call
Command started
#44

shell command

tool result
Command finished
#45

shell command exit code 0

tool call
Command started
#46

shell command

tool result
Command finished
#47

shell command exit code 0

tool call
Command started
#48

shell command

patch written
Patch captured
#49

Flux captured agent.patch for this trial

validation
Tests passed
#50

go

equivalence
Equivalence judgment
#51

non_equivalent

code review
Code review judgment
#52

fail

decision
Final decision
#53

pass_with_warn

Quality

equivalence
non_equivalent
88% confidence
code review
fail
3 findings
footprint
medium (0.40)
behavioral
100.0%
cost
$2.94 · 10.6M

Equivalence Reasoning

behavioral

The patch captures part of the intent (spec-like messages, added locations, centralized status/extension assignment), but it misses core behavior. In field-selection validation it still reports leaf-selection errors with the field name as the type (e.g., using `definitionName` instead of the actual GraphQL type like `String`), and it uses field position rather than selection-set brace position for that error location. It also removes validator options entirely, which likely breaks existing Apollo-compatibility flag flow (notably `ReplaceInvalidVarError`) instead of centralizing while preserving other flag behavior.

Code Review

correctness: 1/4edge case handling: 1/4introduced bug risk: 1/4maintainability idioms: 2/4

The patch passes tests but likely does not satisfy the intended change set: it removes configurable compatibility plumbing, applies validation metadata too broadly, and still misses key spec-format error details.

3 findings
Apollo compatibility flag migration is incomplete and API was removed
major

The change removes `ReplaceUndefinedOpFieldError` and also removes validator option plumbing entirely, but does not introduce the new `UseGraphQLValidationFailedStatus` compatibility flag pathway. This diverges from the intended configurable centralized mechanism and changes public validator construction behavior.

v2/pkg/apollocompatibility/flags.go:3
Validation metadata is now injected globally at walker level
major

The walker now assigns `GRAPHQL_VALIDATION_FAILED` and HTTP 400 for any external error missing those fields. This is broader than validation-rule errors and is not gated by compatibility options, creating behavior drift and potential unintended status/code changes.

v2/pkg/astvisitor/visitor.go:3923
Field selection error text/location handling does not fully match target spec format
major

Missing-subselection errors omit the expected `Did you mean "X { ... }"?` suffix, and leaf-selection errors use generic field position-derived locations rather than dedicated selection-set brace locations, so output is still inconsistent with the intended canonical formatting.

v2/pkg/operationreport/externalerror.go:163