# Which coding agent is best on your repo? | Stet

Stet publishes a public search API over its indexed pages; the guide is at https://www.stet.sh/agents/agent-access.

Source: https://www.stet.sh/private
Source observed: 2026-11-19T11:08:22.164Z
Last checked: 2026-11-19T11:08:22.164+00:00
Indexed: 2026-11-19T11:18:58.127+00:00

---

# Which coding agent is best on your repo?

Public benchmarks rank models on someone else’s code. We replay your merged pull requests, run each setup on the same tasks, and tell you which one to use. Your first trial is free and uses your own login.

Next, Ben replies and we work on getting you set up. You can also [book a demo](https://calendar.app.google/u3ZJ9vcN9zF7TFpV9).

![Theodolite line illustration](https://www.stet.sh/_next/image?url=%2Fdrafts%2Fart%2Fprivate%2Ftheodolite-line.png&w=3840&q=75&dpl=dpl_BwM6xKmWcZsBDH8WGS5G3MFv7DBQ)

## You’ll get a private report like these

The following are public example reports, each run on 24 tasks from our own repository. They are examples, not your results.

| Date | Comparison | Description | Won–tied–lost · $/task |
|---|---|---|---|
| Oct 12 2026 | [GPT 6.1 Sol vs GPT 6 Sol](https://www.stet.sh/blog/gpt-6-1-sol-vs-gpt-6-sol) | A cheaper replacement that holds quality. | 9-10-5 · $1.10 vs $3.01 |
| Oct 12 2026 | [GPT 6.1 Sol vs Grok 4.7 High](https://www.stet.sh/blog/gpt-6-1-sol-vs-gpt-6-sol#leaderboard) | Edges Grok at about a fifth of the cost. | 10-5-9 · $1.10 vs $5.76 |
| Oct 12 2026 | [GPT 6 Luna, max vs low](https://www.stet.sh/blog/gpt-6-1-sol-vs-gpt-6-sol#leaderboard) | Max clearly beats low at about 7× the cost. | 20-2-2 · $0.37 vs $0.05 |
| Oct 12 2026 | [Haiku 5.5, max vs low](https://www.stet.sh/blog/gpt-6-1-sol-vs-gpt-6-sol#leaderboard) | Max edges low at about 15× the cost. | 12-4-8 · $3.85 vs $0.25 |

## Where your code goes

Your free trial can run **Local** or **Hosted**. We do not publish your results.

### Local

We never receive your code, tests, or logins. The agents call your model provider using your own login, and we receive only your sign-in and a usage log of commands and model names.

### Hosted

We copy the one GitHub repository you select and run its commands and the agent only inside an isolated sandbox. The sandbox can reach the internet.

### Logins

Use a Claude Code login or Anthropic API key, a ChatGPT login for Codex, or a Cursor login for models such as Grok. With Hosted, your login stays outside the sandbox in a relay that forwards the agent’s requests.

### Who can read

Only your organization and the teammates you add can read your reports, traces, and repository snapshot. Our staff can open a run to fix it.

### Deletion

Reports and traces stay until you delete them. You can delete one run or all of your organization’s data at any time. We then purge the code, reports, and logins from storage, our database, and sandbox snapshots.

### Compliance

We do not yet have a SOC 2 report or DPA. If you need one first, run Local.

## Questions

### What is a private coding-agent eval?

It measures coding agents on your own repository instead of a public benchmark, like a private SWE-bench built from your merged pull requests. Each task is a merged change, with the repository as it was just before it merged, a prompt, and the change’s own tests, which the agent never sees. We run each setup on the same tasks and send you a report comparing quality, cost, and time.

### Can I compare Claude Code, Codex, and Cursor on my own repo?

Yes. Each setup is an agent with its model and reasoning effort: Claude Code, Codex, or Cursor’s agent with Composer or Grok. Every setup receives the same tasks in its own container, built from your repository with dependencies installed, and runs on the Claude, Codex, or Cursor login you already have.

### Can I test whether my `AGENTS.md` or `CLAUDE.md` helps?

Yes. Your current instructions and the changed version run as two setups on the same tasks. The report shows how the change moved correctness, review, cost, and time before it reaches every agent session.

### How is grading different from passing tests?

Tests are one signal. We also judge each patch against the task’s required behaviors, written without seeing any agent’s patch, and review it for code reuse, abstraction, test style, dead code, and scope: would a senior engineer merge it? Correctness, checklist, and review combine into one score per patch, alongside tokens, estimated cost, and minutes per task.

### Where does my code run?

You choose. With Local, agents work on your machine and call your model provider using your own login, so we never receive your code, tests, or logins. With Hosted, we copy the one GitHub repository you select into an isolated sandbox and keep your login outside it. We do not publish your results.

---

[Read this page’s summary](https://www.stet.sh/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fprivate&trace=t_5da45423e484)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Stet’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.stet.sh/agents/agent/faq?q={question}&trace=t_5da45423e484`

[Tool guide and examples](https://www.stet.sh/agents/agent-access) · [OpenAPI](https://www.stet.sh/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.stet.sh/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.stet.sh%2Fprivate). Opening a form does not execute a tool.