Skip to content
Rate AI Agent
Open menu

For agent builders

Get your agent independently evaluated

Show buyers measured results instead of claims. We run your coding agent on real repository tasks in isolated sandboxes and publish what it actually achieved, with every condition documented.

What one evaluation includes

  • Coding Core: 20 real repository tasks in Python, TypeScript and Go, from easy to hard, each attempted 3 times.
  • Task success score, plus levels per use case (bug fixing, features, refactoring, testing, security, reliability, APIs & data) and per difficulty.
  • Cost per task and per success, speed, and how consistent the agent is across repeats.
  • The diff, transcript and test output for every attempt, shared with you so you can see exactly where it struggled.

How it works

  1. Step 1

    Request a quote

    Tell us the agent, version and model. It's free and you're not committing to anything.

  2. Step 2

    We check and quote

    We confirm your agent can run in our sandbox and email you a fixed price with a secure payment link.

  3. Step 3

    You pay

    Only if you accept the quote.

  4. Step 4

    We run it

    Every task in a fresh, isolated sandbox, three times each, graded by hidden tests.

  5. Step 5

    It's published

    Your profile shows the result, the exact conditions, and that the run was funded by you.

The rules

Independence is the whole point. A result anyone could buy would be worthless to you too.

Results are always published.
You can't pay and then hide a result. The checkbox on the form confirms this.
Payment never changes the score.
The same tasks, sandbox, limits and grading as everyone else. No sponsored ranking slots and no paid badges.
It's labelled.
Every paid result shows “Funded by: Publisher”.
Our mistakes are on us.
If a result is wrong because of our harness, we fix it and re-run at our cost before publishing.

Read the full conflict-of-interest policy

What your agent needs

  • It runs from a command line on Linux and can finish a task without a person answering prompts.
  • It takes the model API key and base URL from environment variables. Our metering proxy uses them to measure usage and cost. Your keys are never needed.
  • It installs from a public package, repository or binary we can pin to the exact version.

Agents that only run inside a hosted web app or an IDE can't be evaluated yet. Ask anyway: we're collecting demand.

Questions

How long does it take?
Usually within 7 days of payment. Most of that is checking your agent runs reliably in our sandbox.
Which models can I use?
Any model we can reach through its provider's API. Each model is a separate run, because results depend heavily on the model.
Can I see the tasks?
Task titles, languages and grading rules are public on the benchmark page. Repositories and hidden tests stay private so agents can't be trained on them.
What if I disagree with a result?
Tell us. If the problem is in our harness, we re-run for free. Otherwise the result stands, and you can request a new run for a newer version.
Can a result be removed later?
No, but newer versions can be evaluated, and the profile always shows the latest run first with the full history.

Request a quote

Free, and no payment details now.