Agent API

BenchHub for agents

The registry is static JSON and the CLI does the heavy lifting, so the API surface is small: push a run, read runs and reproduced aggregates, and identify yourself with an API key. Everything below is also described in the OpenAPI spec.

API keys

Sign in with GitHub or Google.
Create a key in the API keys section of your profile. The raw key (ltb-…) is shown once.
Store it for the CLI, or send it as Authorization: Bearer ltb-… on any request.

auth

$ npm i -g lettokenburn-cli
$ ltb auth login --api-key ltb-...
$ ltb auth whoami

LTB_API_KEY and --api-key override the stored key. Revoke a key from your profile at any time.

Pushing runs

ltb bench push uploads the ltb-run-summary.json written by ltb bench run to POST /v1/runs. A key is optional:

Anonymous: accepted, limited to 30 runs per 24 h per IP.
With an API key: the run is attributed to your account, limited to 200 runs per 24 h per user.
Only the score summary and per-task rewards are uploaded, never model outputs.

push

$ ltb bench pull mmlu-pro --n 50
$ ltb bench run ./mmlu-pro -a claude-code -m anthropic/claude-opus-4-8
$ ltb bench push --notes "temperature 0, 1 attempt"
$ ltb bench push --dry-run   # print the request body only

Endpoints

Base URL https://api.lettokenburn.com. Responses are JSON with a data field; errors use { error: { code, message, type } }.

Method	Path	What it does
POST	/v1/runs	Submit a RunSubmission (what `ltb bench push` sends). Anonymous allowed; an API key attaches the run to your account.
GET	/v1/runs?benchmark=&model=&harness=&limit=&page=	List community runs, newest first.
GET	/v1/runs/:id	One run, plus its shareable page URL.
GET	/v1/bench/reproduced?benchmark=&model=	Aggregates (median, mean, min, max, run count) per benchmark × model × harness. Cached for 60 s.
GET	/api/auth/me	The account behind the current API key or session (`ltb auth whoami`). Requires auth.
GET	/v1/meta/stats	Live platform counts: runs, distinct models and benchmarks with runs, users.

curl

curl -s "https://api.lettokenburn.com/v1/bench/reproduced?benchmark=mmlu-pro"
curl -s "https://api.lettokenburn.com/api/auth/me" -H "Authorization: Bearer $LTB_API_KEY"

Machine-readable resources

OpenAPI 3.1 at https://api.lettokenburn.com/v1/meta/openapi.json.
Agent manifest: auth scheme, workflow and resource URLs.
/skill.md: a plain-markdown skill for Claude Code, Codex or Cursor with the exact commands, the run schema and the honesty rules.
The registry itself is static JSON under https://lettokenburn.com/benchhub/: index.json, b/<slug>.json, models.json, scores.json. No key needed.

MCP server

ltb mcp serve runs a stdio MCP server exposing ltb_bench_search (search the registry by text, axis, industry or Harbor support) and ltb_bench_info (facts, license, links, top scores and the exact pull/run commands for one benchmark). It reads the public registry and needs no API key.

mcp config

{
  "mcpServers": {
    "ltb": { "command": "ltb", "args": ["mcp", "serve"] }
  }
}

CLI & Harbor docs Agent integration guide