STET

Run it on your repo.

We set up your first trial free. It runs on your own Claude, Codex or Cursor login.

Next, Ben replies and we work on getting you set up.

or book a demo

Where your code goes.

Your free trial can run Local or Hosted. Stet doesn’t publish your results.

Local
Stet never receives your code, tests or logins. The agents call your model provider on your own login, and Stet gets only your sign-in and a usage log of commands and model names.
Hosted
Stet copies the one GitHub repo you pick and runs its commands and the agent only inside an isolated sandbox. The sandbox can reach the internet.
Logins
Use a Claude Code login or Anthropic API key, a ChatGPT login for Codex, or a Cursor login for models such as Grok. On Hosted, your login stays outside the sandbox, in a relay that forwards the agent’s requests.
Who can read
Only your org and the teammates you add can read your reports, traces and repo snapshot. Stet staff can open a run to fix it.
Deletion
Reports and traces stay until you delete them. Delete one run or all of your org’s data at any time, and Stet purges the code, reports and logins from storage, its database and sandbox snapshots.
Compliance
Stet has no SOC 2 report or DPA yet. If you need one first, run Local.

You’ll get a private report like these.

These are public examples, each run on 24 tasks from Stet’s own repo.

won-tied-lost · $ / task

  1. GPT 6.1 Sol vs GPT 6 SolA cheaper replacement that holds quality.9-10-5 · $1.10 vs $3.01
  2. GPT 6.1 Sol vs Grok 4.7 HighEdges Grok at about a fifth of the cost.10-5-9 · $1.10 vs $5.76
  3. GPT 6 Luna, max vs lowMax clearly beats low at about 7× the cost.20-2-2 · $0.37 vs $0.05
  4. Haiku 5.5, max vs lowMax edges low at about 15× the cost.12-4-8 · $3.85 vs $0.25

Questions.

What is a private coding-agent eval?

It measures coding agents on your own repository instead of a public benchmark, like a private SWE-bench built from your merged PRs. Each task is a merged change, with the repo as it was just before it merged, a prompt and the change’s own tests, which the agent never sees. Stet runs each setup on the same tasks and sends you a report comparing quality, cost and time.

Can I compare Claude Code, Codex and Cursor on my own repo?

Yes. Each setup is an agent with its model and reasoning effort: Claude Code, Codex, or Cursor’s agent with Composer or Grok. Every setup gets the same tasks, each in its own container built from your repo with its dependencies installed, and runs on the Claude, Codex or Cursor login you already have.

Can I test whether my AGENTS.md or CLAUDE.md helps?

Yes. Your current instructions and the changed version run as two setups on the same tasks, so the report shows how the change moved correctness, review, cost and time before it reaches every agent session.

How is grading different from passing tests?

Tests are one signal. Stet also judges each patch against the task’s required behaviors, written without seeing any agent’s patch, and reviews it for code reuse, abstraction, test style, dead code and scope: would a senior engineer merge it? Correctness, checklist and review combine into one score per patch, alongside tokens, estimated cost and minutes per task.

Where does my code run?

Your choice. Run Local and the agents work on your machine and call your model provider on your own login, so Stet never receives your code, tests or logins. Run Hosted and Stet copies the one GitHub repo you pick into an isolated sandbox, keeping your login outside it. Stet doesn’t publish your results.

stetsh