Natural Language Inference on TERRa

Metric: Accuracy (higher is better)

LeaderboardDataset

Loading chart...

Results

Submit a result

Sort:

#	Model↕	Accuracy▼	Extra Data	Paper	Date↕	Code
1	Human Benchmark	0.92	No	RussianSuperGLUE: A Russian Language Understandi...	2020-10-29	Code
2	Golden Transformer	0.871	No	-	-	-
3	ruRoberta-large finetune	0.801	No	-	-	-
4	ruT5-large-finetune	0.747	No	-	-	-
5	ruBert-large finetune	0.704	No	-	-	-
6	ruBert-base finetune	0.703	No	-	-	-
7	ruT5-base-finetune	0.692	No	-	-	-
8	RuGPT3Large	0.654	No	-	-	-
9	RuBERT plain	0.642	No	-	-	-
10	RuBERT conversational	0.64	No	-	-	-
11	SBERT_Large_mt_ru_finetuning	0.637	No	-	-	-
12	SBERT_Large	0.637	No	-	-	-
13	Multilingual Bert	0.617	No	-	-	-
14	YaLM 1.0B few-shot	0.605	No	-	-	-
15	RuGPT3XL few-shot	0.573	No	-	-	-
16	MT5 Large	0.561	No	mT5: A massively multilingual pre-trained text-t...	2020-10-22	Code
17	heuristic majority	0.549	No	Unreasonable Effectiveness of Rule-Based Heurist...	2021-05-03	-
18	majority_class	0.513	No	Unreasonable Effectiveness of Rule-Based Heurist...	2021-05-03	-
19	RuGPT3Medium	0.505	No	-	-	-
20	RuGPT3Small	0.488	No	-	-	-
21	Random weighted	0.483	No	Unreasonable Effectiveness of Rule-Based Heurist...	2021-05-03	-
22	Baseline TF-IDF1.1	0.471	No	RussianSuperGLUE: A Russian Language Understandi...	2020-10-29	Code

#1Human BenchmarkSOTA
0.92
Accuracy· 2020-10-29
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark Code
#2Golden Transformer
0.871
Accuracy
No paper
#3ruRoberta-large finetune
0.801
Accuracy
No paper
#4ruT5-large-finetune
0.747
Accuracy
No paper
#5ruBert-large finetune
0.704
Accuracy
No paper
#6ruBert-base finetune
0.703
Accuracy
No paper
#7ruT5-base-finetune
0.692
Accuracy
No paper
#8RuGPT3Large
0.654
Accuracy
No paper
#9RuBERT plain
0.642
Accuracy
No paper
#10RuBERT conversational
0.64
Accuracy
No paper
#11SBERT_Large_mt_ru_finetuning
0.637
Accuracy
No paper
#12SBERT_Large
0.637
Accuracy
No paper
#13Multilingual Bert
0.617
Accuracy
No paper
#14YaLM 1.0B few-shot
0.605
Accuracy
No paper
#15RuGPT3XL few-shot
0.573
Accuracy
No paper
#16MT5 LargeSOTA
0.561
Accuracy· 2020-10-22
mT5: A massively multilingual pre-trained text-to-text transformer Code
#17heuristic majority
0.549
Accuracy· 2021-05-03
Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks
#18majority_class
0.513
Accuracy· 2021-05-03
Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks
#19RuGPT3Medium
0.505
Accuracy
No paper
#20RuGPT3Small
0.488
Accuracy
No paper
#21Random weighted
0.483
Accuracy· 2021-05-03
Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks
#22Baseline TF-IDF1.1
0.471
Accuracy· 2020-10-29
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark Code