# Test AGENTS.md Changes Before They Reach Every Agent

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/instruction-evals
Source observed: 2026-11-15T21:03:04.891Z
Last checked: 2026-11-16T02:17:24.27+00:00
Indexed: 2026-11-15T21:06:06.869+00:00

---

# Test AGENTS.md changes before they reach every agent

We test coding-agent instruction changes on real repository work before they become shared AI infrastructure. Our instruction evaluations help teams determine both whether a change avoids regressions and whether it improves results.

[Run an instruction eval on your repo](https://www.stet.sh/private) · [Read the AGENTS.md testing story](https://www.stet.sh/blog/agents-md-testing)

## Why instructions need evals

Agent-native organizations may write up to 100% of their code with AI, yet still ship `AGENTS.md` changes based on intuition. One bad line can affect every agent session for every developer in a repository. We provide a way to move from intuition to rigor.

## What can go wrong

Adding too much implementation detail to `AGENTS.md` instead of using progressive disclosure can confuse an agent on unrelated tasks. It may then ignore other instructions in the file.

## What we compare

- `AGENTS.md` changes
- Model comparisons, such as Opus 4.7 versus 4.6
- Model reasoning effort
- Harness changes, such as Claude in Cursor versus Claude in Claude Code
- `SKILL.md` changes

## What our eval measures

### Correctness

We run the same tests and validation commands that make the original repository trustworthy.

### Equivalence

We check whether the patch solves the same problem, rather than merely changing nearby files.

### Review quality

We score maintainability, bug risk, edge cases, coherence, and instruction adherence beyond the test gate.

### Footprint risk

We measure whether the candidate made the patch larger, touched unrelated areas, or created merge risk.

### Cost and traces

We keep token spend, attempts, logs, and supporting evidence attached to the result so the rollout decision is inspectable.

## How it works

### Build the candidate

Point us at the current instruction setup and the candidate: `AGENTS.md`, `SKILL.md`, model, reasoning effort, or harness.

### Replay real work

We select tasks from repository history, reset to the pre-change state, and run each arm through the same agent workflow.

### Read the rollout signal

The result shows what improved, what regressed, what changed in cost, and whether the evidence is strong enough to ship.

The same [methodology](https://www.stet.sh/methodology) powers our model comparisons and private-repository evaluations. Use it before an instruction, model, reasoning, skill, or harness change becomes the default path for every agent session.

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Finstruction-evals&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Finstruction-evals). Opening a form does not execute a tool.