LetTokenBurn
Sign in
Auto71%Harbor: pendingAgentic / Tool Use

LiveMACEBench

LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets

A process-aware benchmark evaluating persistent LLM agents in live cryptocurrency and U.S. equity markets over 30 days using outcome and mechanism-specific diagnostics.

PaperRepo

How to run

ltb · livemacebench
$ npm i -g lettokenburn-cli
$ ltb bench info livemacebench

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

5 sourced entries
HarnessSourceConfidenceLink
Gemini 3.1 Pro0.9N/AIndependentmediumOct 7, 2026arxiv.org
Grok 4.200.8N/AIndependentmediumOct 7, 2026arxiv.org
GPT-5.40.7N/AIndependentmediumOct 7, 2026arxiv.org
DeepSeek V3.20.6N/AIndependentmediumOct 7, 2026arxiv.org
Qwen3 Max (2025-09-23)0.5N/AIndependentmediumOct 7, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

LiveMACEBench uses live financial markets as an evolving testbed for five frontier LLMs operating in Base, Tool Use, Persistent Memory, Rule Following, and Multi-Agent Collaboration configurations. It evaluates agents through realized outcomes and trace-level process diagnostics to distinguish capability from performance.

Scoring combines task outcomes (returns) with controlled mechanism effects and process diagnostics. Results reveal an outcome–capability gap, where return rankings diverge from capability measurements, exposing bottlenecks like ineffective memory reuse and redundant tool interactions.