Common Sense Reasoning on RWSD

Metric: Accuracy (higher is better)

LeaderboardDataset

Loading chart...

Results

Submit a result

Sort:

#	Model↕	Accuracy▼	Extra Data	Paper	Date↕	Code
1	Human Benchmark	0.84	No	RussianSuperGLUE: A Russian Language Understandi...	2020-10-29	Code
2	SBERT_Large_mt_ru_finetuning	0.675	No	-	-	-
3	ruT5-large-finetune	0.669	No	-	-	-
4	ruT5-base-finetune	0.669	No	-	-	-
5	ruBert-large finetune	0.669	No	-	-	-
6	ruBert-base finetune	0.669	No	-	-	-
7	YaLM 1.0B few-shot	0.669	No	-	-	-
8	MT5 Large	0.669	No	mT5: A massively multilingual pre-trained text-t...	2020-10-22	Code
9	RuBERT plain	0.669	No	-	-	-
10	RuBERT conversational	0.669	No	-	-	-
11	Multilingual Bert	0.669	No	-	-	-
12	heuristic majority	0.669	No	Unreasonable Effectiveness of Rule-Based Heurist...	2021-05-03	-
13	RuGPT3Medium	0.669	No	-	-	-
14	RuGPT3Small	0.669	No	-	-	-
15	majority_class	0.669	No	Unreasonable Effectiveness of Rule-Based Heurist...	2021-05-03	-
16	SBERT_Large	0.662	No	-	-	-
17	Baseline TF-IDF1.1	0.662	No	RussianSuperGLUE: A Russian Language Understandi...	2020-10-29	Code
18	RuGPT3XL few-shot	0.649	No	-	-	-
19	RuGPT3Large	0.636	No	-	-	-
20	Random weighted	0.597	No	Unreasonable Effectiveness of Rule-Based Heurist...	2021-05-03	-
21	ruRoberta-large finetune	0.571	No	-	-	-
22	Golden Transformer	0.545	No	-	-	-

#1Human BenchmarkSOTA
0.84
Accuracy· 2020-10-29
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark Code
#2SBERT_Large_mt_ru_finetuning
0.675
Accuracy
No paper
#3ruT5-large-finetune
0.669
Accuracy
No paper
#4ruT5-base-finetune
0.669
Accuracy
No paper
#5ruBert-large finetune
0.669
Accuracy
No paper
#6ruBert-base finetune
0.669
Accuracy
No paper
#7YaLM 1.0B few-shot
0.669
Accuracy
No paper
#8MT5 LargeSOTA
0.669
Accuracy· 2020-10-22
mT5: A massively multilingual pre-trained text-to-text transformer Code
#9RuBERT plain
0.669
Accuracy
No paper
#10RuBERT conversational
0.669
Accuracy
No paper
#11Multilingual Bert
0.669
Accuracy
No paper
#12heuristic majority
0.669
Accuracy· 2021-05-03
Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks
#13RuGPT3Medium
0.669
Accuracy
No paper
#14RuGPT3Small
0.669
Accuracy
No paper
#15majority_class
0.669
Accuracy· 2021-05-03
Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks
#16SBERT_Large
0.662
Accuracy
No paper
#17Baseline TF-IDF1.1
0.662
Accuracy· 2020-10-29
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark Code
#18RuGPT3XL few-shot
0.649
Accuracy
No paper
#19RuGPT3Large
0.636
Accuracy
No paper
#20Random weighted
0.597
Accuracy· 2021-05-03
Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks
#21ruRoberta-large finetune
0.571
Accuracy
No paper
#22Golden Transformer
0.545
Accuracy
No paper