LetTokenBurn
Sign in
Auto86%Harbor adapterMultilingualKnowledge & Reasoning

XTREME

Cross-Lingual Transfer Evaluation of Multilingual Encoders

The Cross-lingual TRansfer Evaluation of Multilingual Encoders (XTREME) benchmark was introduced to encourage more research on multilingual transfer learning,.

RepoDatasetLeaderboardHomepage

How to run

ltb · xtreme
$ npm i -g lettokenburn-cli
$ ltb bench pull xtreme --n 50
$ ltb bench run ./xtreme -a claude-code -m anthropic/claude-opus-4-8
$ ltb bench push

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

1 sourced entry
HarnessSourceConfidenceLink
ERNIE-M80.9N/AIndependentmediumDec 31, 2020arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

The Cross-lingual TRansfer Evaluation of Multilingual Encoders (XTREME) benchmark was introduced to encourage more research on multilingual transfer learning,. XTREME covers 40 typologically diverse languages spanning 12 language families and includes 9 tasks that require reasoning about different levels of syntax or semantics.

The languages in XTREME are selected to maximize language diversity, coverage in existing tasks, and availability of training data. The languages in XTREME are selected to maximize language diversity, coverage in existing tasks, and availability of training data. Among these are many under-studied languages, such as the Dravidian languages Tamil (spoken in southern India, Sri Lanka, and Singapore), Telugu and Malayalam (spoken mainly in southern India), and the Niger-Congo languages Swahili and Yoruba, spoken in Africa.