LetTokenBurnOpenGameEval: Benchmarking Agentic Programming and Exploration in a Stateful Game Engine
A benchmark for agentic game development in Roblox Studio with 84 core tasks, measuring pass rates and exploration behavior of LLM agents.
$ npm i -g lettokenburn-cli$ ltb bench info opengameeval
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Claude Fable 5 | 51.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Gemini 3 Flash Preview | 51.6 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Claude Opus 4.8 | 51.5 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Gemini 3.1 Pro | 51.3 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Gemini-3.5-flash | 50.6 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GPT-5.6 Sol | 47.3 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GPT-5.5 | 46.1 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Claude Opus 4.7 | 45.5 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GLM-5 | 42.9 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GPT-5.6-terra | 40.4 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GPT-5.6 Luna | 40.3 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GPT-5.4 | 40.0 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GLM-4.7 | 36.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
OpenGameEval evaluates LLM agents in reproducible Roblox Studio sessions using an eight-tool action space that separates observation from editing. It comprises 84 human-curated tasks across 11 curated experiences, scored via executable checks on edited scenes and simulated play sessions.
The benchmark measures both task success and exploration coverage against annotated reference solutions. Frontier models achieve ~51.7% pass rate on single attempts, with exploration strongly predicting success, highlighting differences in agent capabilities beyond final outcomes.