MTEB

Massive Text Embedding Benchmark

TextsApache-2.0 licenseIntroduced 2022-10-13

MTEB is a benchmark that spans 8 embedding tasks covering a total of 56 datasets and 112 languages. The 8 task types are Bitext mining, Classification, Clustering, Pair Classification, Reranking, Retrieval, Semantic Textual Similarity and Summarisation. The 56 datasets contain varying text lengths and they are grouped into three categories: Sentence to sentence, Paragraph to paragraph, and Sentence to paragraph.

Check the latest leaderboards at HuggingFace.

Benchmarks

Classification/Accuracy Information Retrieval/nDCG@10 Retrieval/nDCG@10 Semantic Textual Similarity/Spearman Correlation Text Classification/Accuracy Text Clustering/V-Measure Text Summarization/Spearman Correlation