LetTokenBurn
Sign in
Auto100%Harbor adapterMultimodalKnowledge & Reasoning

MapEval-Visual

MapEval-Visual contains 400 image-question-answer triplets.

PaperRepoDatasetLeaderboardHomepage

How to run

ltb · mapeval-visual
$ npm i -g lettokenburn-cli
$ ltb bench pull mapeval-visual --n 50
$ ltb bench run ./mapeval-visual -a claude-code -m anthropic/claude-opus-4-8
$ ltb bench push

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

1 sourced entry
HarnessSourceConfidenceLink
Claude 3.5 Sonnet61.6N/AIndependentmediumDec 31, 2024arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

MapEval-Visual contains 400 image-question-answer triplets. Each question is paired with a snapshot from google maps website. The task is the answer question based on the provided map snapshot.