BBH
Harbor adapterBIG-Bench Hard evaluates advanced reasoning capabilities across 23 challenging tasks using over 6,500 English samples.
Verified100%Knowledge & Reasoning
- Samples
- 6,511
- Released
- 2022
- Scores
- 59
EconLogicQA
EconLogicQA is a benchmark designed to test the sequential reasoning skills of large language models (LLMs) in economics, business, and supply chain management.
How to run
ltb · econlogicqa
$ npm i -g lettokenburn-cli$ ltb bench info econlogicqa
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
9 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-4 | 56.9 | N/A | Independent | medium | May 13, 2024 | arxiv.org |
| GPT-3.5 Turbo | 37.7 | N/A | Independent | medium | May 13, 2024 | arxiv.org |
| LLama-3 8B | 34.6 | N/A | Independent | medium | May 13, 2024 | arxiv.org |
| Mistral-7B-Instruct-v0.2 | 31.5 | N/A | Independent | medium | May 13, 2024 | arxiv.org |
| Mistral-7B-v0.1 | 26.1 | N/A | Independent | medium | May 13, 2024 | arxiv.org |
| LLaMA 2 13B | 14.6 | N/A | Independent | medium | May 13, 2024 | arxiv.org |
| LLaMA 2 7B | 9.2 | N/A | Independent | medium | May 13, 2024 | arxiv.org |
| Gemma-2B-IT | 8.5 | N/A | Independent | medium | May 13, 2024 | arxiv.org |
| Gemma 7B | 2.3 | N/A | Independent | medium | May 13, 2024 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
EconLogicQA is a benchmark designed to test the sequential reasoning skills of large language models (LLMs) in economics, business, and supply chain management. It diverges from typical benchmarks by requiring models to understand and sequence multiple interconnected events, capturing complex economic logics. The benchmark includes multi-event scenarios and a thorough suite of evaluations to assess proficiency in economic contexts.