ARC-AGI-2
Harbor nativeARC-AGI-2: 更难的抽象推理
Verified100%Abstract Reasoning
- Samples
- 167
- Released
- 2025
- Scores
- 118
EvoRiskBench
EvoRiskBench: An Evolving Benchmark for Runtime Security Risks in Workspace Agents
A benchmark of 450 executable adversarial tasks measuring runtime security risks (entry points to effects) in workspace agents across six scenarios.
How to run
ltb · evoriskbench
$ npm i -g lettokenburn-cli$ ltb bench info evoriskbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
3 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Claude Opus 5 | 43.4 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| DeepSeek V4 Pro 0813 | 68.2 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| GPT-5.6 Sol | 69.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
EvoRiskBench contains 450 executable adversarial tasks across six scenarios, structured by an EP–Path–EF framework linking nine risk entry points to five technical effects. Cases are constructed via an automated workflow and verified using runtime traces and environment states in isolated Windows-based Docker environments.
Evaluation uses Attack Success Rate (ASR), Task Success Rate, and Harm Score across nine Model × Harness configurations. The highest ASR reached 68.44%, revealing substantial vulnerabilities where stronger task execution coincided with greater attack susceptibility.