LetTokenBurn
Sign in
Auto71%Harbor: pendingSafety & AlignmentAgentic / Tool Use

MADBench

MADBench: Benchmarking the Security of Multi-Agent Debate

A benchmark evaluating the security and robustness of multi-agent debate systems against six attack families across question-answering and workspace tasks.

Paper

How to run

ltb · madbench
$ npm i -g lettokenburn-cli
$ ltb bench info madbench

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

3 sourced entries
HarnessSourceConfidenceLink
GPT-6 Astra50.6N/AIndependentmediumSep 30, 2026arxiv.org
GPT-562.2N/AIndependentmediumSep 30, 2026arxiv.org
GPT-4o73.1N/AIndependentmediumSep 30, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

Content & Construction: MADBench evaluates the security of Multi-Agent Debate (MAD) by organizing attacks into a layered taxonomy following the MAD workflow. It incorporates four single-agent attacks (direct/indirect prompt injection, RAG poisoning, tool hijacking) and two multi-agent specific attacks (coordination among compromised agents, compromised orchestrator). The benchmark uses AutoGen AgentChat to implement evaluations on 356 source tasks comprising 3,958 security test cases.

Scoring & Key Findings: It measures attack absorption or amplification using an Amplification Factor (AF) metric across goals like accuracy, cost, incorrect read, and incorrect write. Results indicate that while MAD can absorb attacks on answer accuracy in question-answering tasks, it often amplifies unauthorized reads/writes in workspace tasks.