Semantic Textual Similarity on STS Benchmark

Metric: Spearman Correlation (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	Spearman Correlation▼	Extra Data	Paper	Date↕	Code
1	Mnet-Sim	0.931	No	MNet-Sim: A Multi-layered Semantic Similarity Ne...	2021-11-09	-
2	MT-DNN-SMART	0.925	No	SMART: Robust and Efficient Fine-Tuning for Pre-...	2019-11-08	Code
3	StructBERTRoBERTa ensemble	0.924	No	StructBERT: Incorporating Language Structures in...	2019-08-13	-
4	T5-11B	0.921	No	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
5	RealFormer	0.8988	No	RealFormer: Transformer Likes Residual Attention	2020-12-21	Code
6	T5-3B	0.898	No	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
7	AnglE-LLaMA-13B	0.8969	No	AnglE-optimized Text Embeddings	2023-09-22	Code
8	ASA + RoBERTa	0.892	No	Adversarial Self-Attention for Language Understa...	2022-06-25	Code
9	PromptEOL+CSE+LLaMA-30B	0.8914	No	Scaling Sentence Embeddings with Large Language ...	2023-07-31	Code
10	AnglE-LLaMA-7B	0.8897	No	AnglE-optimized Text Embeddings	2023-09-22	Code
11	AnglE-LLaMA-7B-v2	0.8897	No	AnglE-optimized Text Embeddings	2023-09-22	Code
12	T5-Large 770M	0.886	No	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
13	PromptEOL+CSE+OPT-13B	0.8856	No	Scaling Sentence Embeddings with Large Language ...	2023-07-31	Code
14	PromptEOL+CSE+OPT-2.7B	0.8833	No	Scaling Sentence Embeddings with Large Language ...	2023-07-31	Code
15	PromCSE-RoBERTa-large (0.355B)	0.8787	No	Improved Universal Sentence Embeddings with Prom...	2022-03-14	Code
16	BigBird	0.878	No	Big Bird: Transformers for Longer Sequences	2020-07-28	Code
17	SimCSE-RoBERTalarge	0.867	No	SimCSE: Simple Contrastive Learning of Sentence ...	2021-04-18	Code
18	Trans-Encoder-RoBERTa-large-cross (unsup.)	0.867	No	Trans-Encoder: Unsupervised sentence-pair modell...	2021-09-27	Code
19	Trans-Encoder-RoBERTa-large-bi (unsup.)	0.8655	No	Trans-Encoder: Unsupervised sentence-pair modell...	2021-09-27	Code
20	BERT-LARGE	0.865	No	BERT: Pre-training of Deep Bidirectional Transfo...	2018-10-11	Code
21	ASA + BERT-base	0.865	No	Adversarial Self-Attention for Language Understa...	2022-06-25	Code
22	Trans-Encoder-BERT-large-bi (unsup.)	0.8616	No	Trans-Encoder: Unsupervised sentence-pair modell...	2021-09-27	Code
23	SRoBERTa-NLI-STSb-large	0.8615	No	Sentence-BERT: Sentence Embeddings using Siamese...	2019-08-27	Code
24	T5-Small	0.85	No	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
25	SBERT-STSb-base	0.8479	No	Sentence-BERT: Sentence Embeddings using Siamese...	2019-08-27	Code
26	Trans-Encoder-RoBERTa-base-cross (unsup.)	0.8465	No	Trans-Encoder: Unsupervised sentence-pair modell...	2021-09-27	Code
27	SBERT-STSb-large	0.8445	No	Sentence-BERT: Sentence Embeddings using Siamese...	2019-08-27	Code
28	FNet-Large	0.84	No	FNet: Mixing Tokens with Fourier Transforms	2021-05-09	Code
29	Trans-Encoder-BERT-base-bi (unsup.)	0.839	No	Trans-Encoder: Unsupervised sentence-pair modell...	2021-09-27	Code
30	Pearl	0.7981	No	-	-	-
31	SBERT-NLI-large	0.79	No	Sentence-BERT: Sentence Embeddings using Siamese...	2019-08-27	Code
32	Mirror-RoBERTa-base (unsup.)	0.787	No	Fast, Effective, and Self-Supervised: Transformi...	2021-04-16	Code
33	Dino (STSb/̄🦕)	0.7782	No	Generating Datasets with Pretrained Language Mod...	2021-04-15	Code
34	SRoBERTa-NLI-base	0.7777	No	Sentence-BERT: Sentence Embeddings using Siamese...	2019-08-27	Code
35	SBERT-NLI-base	0.7703	No	Sentence-BERT: Sentence Embeddings using Siamese...	2019-08-27	Code
36	Dino (STS/̄🦕)	0.7651	No	Generating Datasets with Pretrained Language Mod...	2021-04-15	Code
37	Mirror-BERT-base (unsup.)	0.764	No	Fast, Effective, and Self-Supervised: Transformi...	2021-04-16	Code
38	BERTlarge-flow (target)	0.7226	No	On the Sentence Embeddings from Pre-trained Lang...	2020-11-02	Code
39	IS-BERT-NLI	0.6921	No	An Unsupervised Sentence Embedding Method by Mut...	2020-09-25	Code
40	Rematch	0.6652	No	Rematch: Robust and Efficient Matching of Local ...	2024-04-02	Code
41	Def2Vec	0.6372	No	-	-	Code

#1Mnet-SimSOTA
0.931
Spearman Correlation· 2021-11-09
MNet-Sim: A Multi-layered Semantic Similarity Network to Evaluate Sentence Similarity
#2MT-DNN-SMARTSOTA
0.925
Spearman Correlation· 2019-11-08
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization Code
#3StructBERTRoBERTa ensembleSOTA
0.924
Spearman Correlation· 2019-08-13
StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding
#4T5-11B
0.921
Spearman Correlation· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#5RealFormer
0.8988
Spearman Correlation· 2020-12-21
RealFormer: Transformer Likes Residual Attention Code
#6T5-3B
0.898
Spearman Correlation· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#7AnglE-LLaMA-13B
0.8969
Spearman Correlation· 2023-09-22
AnglE-optimized Text Embeddings Code
#8ASA + RoBERTa
0.892
Spearman Correlation· 2022-06-25
Adversarial Self-Attention for Language Understanding Code
#9PromptEOL+CSE+LLaMA-30B
0.8914
Spearman Correlation· 2023-07-31
Scaling Sentence Embeddings with Large Language Models Code
#10AnglE-LLaMA-7B
0.8897
Spearman Correlation· 2023-09-22
AnglE-optimized Text Embeddings Code
#11AnglE-LLaMA-7B-v2
0.8897
Spearman Correlation· 2023-09-22
AnglE-optimized Text Embeddings Code
#12T5-Large 770M
0.886
Spearman Correlation· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#13PromptEOL+CSE+OPT-13B
0.8856
Spearman Correlation· 2023-07-31
Scaling Sentence Embeddings with Large Language Models Code
#14PromptEOL+CSE+OPT-2.7B
0.8833
Spearman Correlation· 2023-07-31
Scaling Sentence Embeddings with Large Language Models Code
#15PromCSE-RoBERTa-large (0.355B)
0.8787
Spearman Correlation· 2022-03-14
Improved Universal Sentence Embeddings with Prompt-based Contrastive Learning and Energy-based Learning Code
#16BigBird
0.878
Spearman Correlation· 2020-07-28
Big Bird: Transformers for Longer Sequences Code
#17SimCSE-RoBERTalarge
0.867
Spearman Correlation· 2021-04-18
SimCSE: Simple Contrastive Learning of Sentence Embeddings Code
#18Trans-Encoder-RoBERTa-large-cross (unsup.)
0.867
Spearman Correlation· 2021-09-27
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations Code
#19Trans-Encoder-RoBERTa-large-bi (unsup.)
0.8655
Spearman Correlation· 2021-09-27
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations Code
#20BERT-LARGESOTA
0.865
Spearman Correlation· 2018-10-11
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Code
#21ASA + BERT-base
0.865
Spearman Correlation· 2022-06-25
Adversarial Self-Attention for Language Understanding Code
#22Trans-Encoder-BERT-large-bi (unsup.)
0.8616
Spearman Correlation· 2021-09-27
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations Code
#23SRoBERTa-NLI-STSb-large
0.8615
Spearman Correlation· 2019-08-27
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Code
#24T5-Small
0.85
Spearman Correlation· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#25SBERT-STSb-base
0.8479
Spearman Correlation· 2019-08-27
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Code
#26Trans-Encoder-RoBERTa-base-cross (unsup.)
0.8465
Spearman Correlation· 2021-09-27
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations Code
#27SBERT-STSb-large
0.8445
Spearman Correlation· 2019-08-27
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Code
#28FNet-Large
0.84
Spearman Correlation· 2021-05-09
FNet: Mixing Tokens with Fourier Transforms Code
#29Trans-Encoder-BERT-base-bi (unsup.)
0.839
Spearman Correlation· 2021-09-27
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations Code
#30Pearl
0.7981
Spearman Correlation
No paper
#31SBERT-NLI-large
0.79
Spearman Correlation· 2019-08-27
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Code
#32Mirror-RoBERTa-base (unsup.)
0.787
Spearman Correlation· 2021-04-16
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders Code
#33Dino (STSb/̄🦕)
0.7782
Spearman Correlation· 2021-04-15
Generating Datasets with Pretrained Language Models Code
#34SRoBERTa-NLI-base
0.7777
Spearman Correlation· 2019-08-27
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Code
#35SBERT-NLI-base
0.7703
Spearman Correlation· 2019-08-27
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Code
#36Dino (STS/̄🦕)
0.7651
Spearman Correlation· 2021-04-15
Generating Datasets with Pretrained Language Models Code
#37Mirror-BERT-base (unsup.)
0.764
Spearman Correlation· 2021-04-16
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders Code
#38BERTlarge-flow (target)
0.7226
Spearman Correlation· 2020-11-02
On the Sentence Embeddings from Pre-trained Language Models Code
#39IS-BERT-NLI
0.6921
Spearman Correlation· 2020-09-25
An Unsupervised Sentence Embedding Method by Mutual Information Maximization Code
#40Rematch
0.6652
Spearman Correlation· 2024-04-02
Rematch: Robust and Efficient Matching of Local Knowledge Graphs to Improve Structural and Semantic Similarity Code
#41Def2Vec
0.6372
Spearman Correlation
No paperCode