AgentMonBench
Harbor: pendingA software-engineering benchmark evaluating monitors' ability to identify consequential agent decisions and locate evidence for user oversight across three subsets.
- Samples
- 300
- Released
- 2026
- Scores
- 8
Australian SOC Benchmark
How Reliable Are Large Language Models for Soil Organic Carbon Assessment? An Australian Benchmark
Evaluates LLMs on 308 Australian paddocks for SOC concentration estimation, stock calculation, and pairwise comparison using measured references.
How to run
ltb · australian-soc-benchmark
$ npm i -g lettokenburn-cli$ ltb bench info australian-soc-benchmark
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
7 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemini 3.8 Flash | 89.0 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| GPT-5.6 Sol | 84.0 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Grok-4.6 | 83.2 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| GPT-5.6-terra | 78.2 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Claude Sonnet 5 | 76.7 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| DeepSeek V4 Pro | 76.1 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Mistral Medium 3.5 | 38.9 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
The benchmark uses 308 Australian paddocks from the SOC-M dataset to test LLMs on three tasks: estimating soil organic carbon concentrations, calculating stocks under varying units, and comparing site pairs. It assesses response validity, dispersion, and cost against measured records or deterministic calculations.
Scoring involves mean absolute error for concentrations, accuracy for stock calculations and pairwise comparisons. Findings show additional soil data reduces error by 13–62%, while unit consistency significantly impacts stock calculation accuracy.