Natural Language Inference on RCB

Metric: Accuracy (higher is better)

LeaderboardDataset

Loading chart...

Results

Submit a result

Sort:

#	Model↕	Accuracy▼	Extra Data	Paper	Date↕	Code
1	Human Benchmark	0.702	No	RussianSuperGLUE: A Russian Language Understandi...	2020-10-29	Code
2	Golden Transformer	0.546	No	-	-	-
3	ruRoberta-large finetune	0.518	No	-	-	-
4	ruBert-base finetune	0.509	No	-	-	-
5	ruBert-large finetune	0.5	No	-	-	-
6	ruT5-large-finetune	0.498	No	-	-	-
7	SBERT_Large_mt_ru_finetuning	0.486	No	-	-	-
8	RuBERT conversational	0.484	No	-	-	-
9	RuGPT3Large	0.484	No	-	-	-
10	majority_class	0.484	No	Unreasonable Effectiveness of Rule-Based Heurist...	2021-05-03	-
11	RuGPT3Small	0.473	No	-	-	-
12	ruT5-base-finetune	0.468	No	-	-	-
13	RuBERT plain	0.463	No	-	-	-
14	RuGPT3Medium	0.461	No	-	-	-
15	MT5 Large	0.454	No	mT5: A massively multilingual pre-trained text-t...	2020-10-22	Code
16	SBERT_Large	0.452	No	-	-	-
17	YaLM 1.0B few-shot	0.447	No	-	-	-
18	Multilingual Bert	0.445	No	-	-	-
19	Baseline TF-IDF1.1	0.441	No	RussianSuperGLUE: A Russian Language Understandi...	2020-10-29	Code
20	heuristic majority	0.438	No	Unreasonable Effectiveness of Rule-Based Heurist...	2021-05-03	-
21	RuGPT3XL few-shot	0.418	No	-	-	-
22	Random weighted	0.374	No	Unreasonable Effectiveness of Rule-Based Heurist...	2021-05-03	-

#1Human BenchmarkSOTA
0.702
Accuracy· 2020-10-29
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark Code
#2Golden Transformer
0.546
Accuracy
No paper
#3ruRoberta-large finetune
0.518
Accuracy
No paper
#4ruBert-base finetune
0.509
Accuracy
No paper
#5ruBert-large finetune
0.5
Accuracy
No paper
#6ruT5-large-finetune
0.498
Accuracy
No paper
#7SBERT_Large_mt_ru_finetuning
0.486
Accuracy
No paper
#8RuBERT conversational
0.484
Accuracy
No paper
#9RuGPT3Large
0.484
Accuracy
No paper
#10majority_class
0.484
Accuracy· 2021-05-03
Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks
#11RuGPT3Small
0.473
Accuracy
No paper
#12ruT5-base-finetune
0.468
Accuracy
No paper
#13RuBERT plain
0.463
Accuracy
No paper
#14RuGPT3Medium
0.461
Accuracy
No paper
#15MT5 LargeSOTA
0.454
Accuracy· 2020-10-22
mT5: A massively multilingual pre-trained text-to-text transformer Code
#16SBERT_Large
0.452
Accuracy
No paper
#17YaLM 1.0B few-shot
0.447
Accuracy
No paper
#18Multilingual Bert
0.445
Accuracy
No paper
#19Baseline TF-IDF1.1
0.441
Accuracy· 2020-10-29
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark Code
#20heuristic majority
0.438
Accuracy· 2021-05-03
Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks
#21RuGPT3XL few-shot
0.418
Accuracy
No paper
#22Random weighted
0.374
Accuracy· 2021-05-03
Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks