UkrMeme
UkrMeme: Multimodal Explainable Persuasion Detection in Ukrainian Memes
A dataset of 200 Ukrainian wartime memes annotated for persuasion techniques to evaluate VLMs in zero-shot multimodal explainable detection.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info ukrmeme
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
4 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-4.1-mini | 0.4 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Qwen3-VL-8B-Instruct | 0.3 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Gemma 3 4B | 0.3 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Qwen3-VL-4B | 0.3 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
UkrMeme comprises 200 organically produced Ukrainian wartime memes annotated with 22 persuasion technique labels plus a 'No Propaganda' class. It complements PTM-UA, a translated subset of the Propaganda 950 benchmark.
Models are evaluated using hierarchical F1 metrics in a zero-shot setup across image, text, and multimodal inputs. The study also analyzes explanation faithfulness by comparing Layer-wise Relevance Propagation with model-generated token attributions.