STET

flux-pr-1308

graphql-go-tools (Go) · W2 · GPT-5.1 Codex Mini

pass_with_warn

Tests passed. 1/1 commands passed. Strength: strong.

96.6% run pass rate
Tier 1
primary testspassednon equivalentfail
go test -C v2 ./... -count=1 -timeout=300s
gold passagent pass

Partial score: 1/1

Publishable: yesCache: miss

Trajectory

codex · partial order only

provider-native trajectory captured; validation and decision steps are appended with coarse ordering only

session start
Session started
#1
tool call
Command started
#2

shell command

tool result
Command finished
#3

shell command exit code 0

tool call
Command started
#4

shell command

tool result
Command finished
#5

shell command exit code 0

tool call
Command started
#6

shell command

tool result
Command finished
#7

shell command exit code 0

tool call
Command started
#8

shell command

tool result
Command finished
#9

shell command exit code 0

tool call
Command started
#10

shell command

tool result
Command finished
#11

shell command exit code 0

tool call
Command started
#12

shell command

tool result
Command finished
#13

shell command exit code 0

tool call
Command started
#14

shell command

tool result
Command finished
#15

shell command exit code 0

tool call
Command started
#16

shell command

tool result
Command finished
#17

shell command exit code 0

tool call
Command started
#18

shell command

tool result
Command finished
#19

shell command exit code 0

tool call
Command started
#20

shell command

tool result
Command finished
#21

shell command exit code 2

tool call
Command started
#22

shell command

tool result
Command finished
#23

shell command exit code 2

tool call
Command started
#24

shell command

tool result
Command finished
#25

shell command exit code 0

tool call
Command started
#26

shell command

tool result
Command finished
#27

shell command exit code 0

tool call
Command started
#28

shell command

tool result
Command finished
#29

shell command exit code 0

tool call
Command started
#30

shell command

tool result
Command finished
#31

shell command exit code 0

tool call
Command started
#32

shell command

tool result
Command finished
#33

shell command exit code 0

tool call
Command started
#34

shell command

tool result
Command finished
#35

shell command exit code 0

tool call
Command started
#36

shell command

tool result
Command finished
#37

shell command exit code 0

tool call
Command started
#38

shell command

tool result
Command finished
#39

shell command exit code 0

tool call
Command started
#40

shell command

tool result
Command finished
#41

shell command exit code 0

tool call
Command started
#42

shell command

tool result
Command finished
#43

shell command exit code 0

tool call
Command started
#44

shell command

tool result
Command finished
#45

shell command exit code 0

tool call
Command started
#46

shell command

tool result
Command finished
#47

shell command exit code 0

tool call
Command started
#48

shell command

patch written
Patch captured
#49

Flux captured agent.patch for this trial

validation
Tests passed
#50

go

equivalence
Equivalence judgment
#51

non_equivalent

code review
Code review judgment
#52

fail

decision
Final decision
#53

pass_with_warn

Quality

equivalence
non_equivalent
85% confidence
code review
fail
4 findings
footprint
medium (0.43)
behavioral
100.0%
cost
$3.27 · 10.9M

Equivalence Reasoning

behavioral

The agent patch appears to only partially implement the feature: it adds `@oneOf` to base schema/fixtures and some variable-side helpers, but does not clearly include the full operation-level oneOf enforcement, nullable-variable-in-oneOf checks, or the undefined-variable error position/reporting changes required by the task. It also seems to change introspection shape (`isOneOf`) rather than clearly exposing the built-in `@oneOf` directive in directive introspection output.

Code Review

correctness: 1/4edge case handling: 1/4introduced bug risk: 1/4maintainability idioms: 2/4

The agent patch likely does not satisfy the intended PR: it includes partial/schema-level edits and some unrelated introspection output changes, but misses core oneOf enforcement and undefined-variable-location behavior needed for correctness.

4 findings
Operation-level oneOf validation is effectively missing
major

The operation values validator only adds a oneOf constant in the shown diff and does not include the required logic to enforce exactly one provided field, non-null value, and nullable-variable rejection at operation validation time.

v2/pkg/astvalidation/operation_rule_values.go:11
Undefined-variable error reporting improvements are not implemented
major

The task requires improved undefined-variable errors with source locations; the shown patch does not update the error-construction API or operation validation flow to carry positions for this case.

v2/pkg/operationreport/externalerror.go:26
Introspection output shape changed in a likely incompatible way
major

The patch adds `isOneOf` to `FullType` and multiple golden outputs, which is a broader schema/output change than required and likely to break existing introspection expectations if not fully wired end-to-end.

v2/pkg/introspection/introspection.go:68
OneOf validation coverage appears incomplete across variable and operation paths
major

Although helper logic was added in variables validation, the patch does not demonstrate the full expected behavior across both runtime variables and operation literals, especially nullable variable usage in oneOf fields.

v2/pkg/variablesvalidation/variablesvalidation.go:21