FG-BMK
Harbor: pendingFG-BMK FG-BMK is a fine-grained evaluation benchmark for large vision-language models and vision-language models.
- Samples
- N/A
- Released
- 2026
- Scores
- 0
FG-OVD
Fine-Grained Open-Vocabulary object Detection benchmarks
Benchmark Suite Description for PapersWithCode Fine-Grained Open-Vocabulary Detection (FG-OVD) Benchmark Suite The FG-OVD benchmark suite evaluates the ability of open-vocabulary object detectors to discern fine-grained object properties such as color, material, pattern, and transparency.
How to run
ltb · fg-ovd
$ npm i -g lettokenburn-cli$ ltb bench info fg-ovd
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
0 sourced entriesNo sourced scores recorded yet.
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Benchmark Suite Description for PapersWithCode
Fine-Grained Open-Vocabulary Detection (FG-OVD) Benchmark Suite The FG-OVD benchmark suite evaluates the ability of open-vocabulary object detectors to discern fine-grained object properties such as color, material, pattern, and transparency. This suite introduces dynamic vocabularies for each object, consisting of one positive caption and several challenging negative captions, crafted using attribute substitution at varying difficulty levels.
Key features include:
- Difficulty-Based Benchmarks: Trivial, Easy, Medium, and Hard benchmarks challenge detectors with progressively harder negative examples.
- Attribute-Based Benchmarks: Focused evaluation of specific attributes, such as color or material, with negative captions differing only in the targeted attribute.
- Metrics: Mean Average Precision (mAP) and Median Rank are used to measure both localization accuracy and fine-grained caption assignment performance.
The suite provides a comprehensive analysis of state-of-the-art models, highlighting their strengths and limitations in fine-grained object recognition.