MYOPIA
MYOPIA: Understanding the Limits of Tree-Based Reasoning in Vision-Language Models
Tree-Based Reasoning in Vision-Language Models MYOPIA is a benchmark for testing whether vision-language models (VLMs) classify images better when they follow a human-designed decision tree, one question at a time, than when they are asked for the label directly (zero-shot).
How to run
$ npm i -g lettokenburn-cli$ ltb bench info myopia
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
3 sourced entriesEvery number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
MYOPIA: Understanding the Limits of Tree-Based Reasoning in Vision-Language Models
MYOPIA is a benchmark for testing whether vision-language models (VLMs) classify images better when they follow a human-designed decision tree, one question at a time, than when they are asked for the label directly (zero-shot). It provides:
Four fixed evaluation subsets: GTSRB (901 images, 43 classes), MNIST (1,000, 10), CIFAR-10 (1,000, 10), and single-object MSCOCO (850, 90).
One decision…