Reading Comprehension on ReClor

Metric: Test (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	Test▼	Extra Data	Paper	Date↕	Code
1	Rational Reasoner / IDOL	80.6	No	IDOL: Indicator-oriented Logic Pre-training for ...	2023-06-27	Code
2	AMR-LE-Ensemble	80	No	-	-	-
3	MERIt(MERIt-deberta-v2-xxlarge )	79.3	No	MERIt: Meta-Path Guided Contrastive Learning for...	2022-03-01	Code
4	MERIt-deberta-v2-xxlarge deberta.v2.xxlarge.path.override_True.norm_1.1.0.w2.A100.cp200.s42	79.3	No	-	-	-
5	Knowledge model	79.2	No	-	-	-
6	DeBERTa-v2-xxlarge-AMR-LE-Contraposition	77.2	No	-	-	-
7	LReasoner ensemble	76.1	No	Logic-Driven Context Extension and Data Augmenta...	2021-05-08	Code
8	ELECTRA and ALBERT	71	No	-	-	-
9	WWZ	69.7	No	-	-	-
10	xlnet-large-uncased [extended data]	69.3	No	-	-	-
11	albert_4gpus_bs2_DACE	68.6	No	-	-	-
12	Tournament2	66.7	No	-	-	-
13	MERIt + Structure Decouple (roberta-large)	65.6	No	-	-	-
14	RoBERTa-single	63.5	No	Logiformer: A Two-Branch Graph Transformer Netwo...	2022-05-02	Code
15	MERIT w/ reasoning aware	63.2	No	-	-	-
16	ALBERT-XXLarge-V2	62.6	No	-	-	-
17	VDGN(RoBERTa single model)	61.7	No	-	-	-
18	ro_DA_CE_v100_224	61.7	No	-	-	-
19	qmcurtis	60.3	No	-	-	-
20	AdaLoGN	60.2	No	-	-	-
21	Zhangjie666	60.2	No	-	-	-
22	alsace	59.2	No	-	-	-
23	RoBERTa-single	58.9	No	Fact-driven Logical Reasoning for Machine Readin...	2021-05-21	Code
24	NAACL 2021	58.2	No	DAGN: Discourse-Aware Graph Network for Logical ...	2021-03-26	Code
25	Roberta Large GCN(dev: 0.63)	57.7	No	-	-	-
26	xlnet_l	57.7	No	-	-	-
27	elBERto	57.3	No	-	-	-
28	5BitClan	56.9	No	-	-	-
29	SimChea	56	No	-	-	-
30	XLNet-large	56	No	ReClor: A Reading Comprehension Dataset Requirin...	2020-02-11	Code
31	RoBERTa-large	55.6	No	ReClor: A Reading Comprehension Dataset Requirin...	2020-02-11	Code
32	XLNet-base	50.4	No	ReClor: A Reading Comprehension Dataset Requirin...	2020-02-11	Code
33	BERT-large+MNLI	50.3	No	-	-	-
34	BERT-large	49.8	No	ReClor: A Reading Comprehension Dataset Requirin...	2020-02-11	Code
35	RoBERTa-base	48.5	No	ReClor: A Reading Comprehension Dataset Requirin...	2020-02-11	Code
36	BERT-base	47.3	No	ReClor: A Reading Comprehension Dataset Requirin...	2020-02-11	Code
37	Gariscat	40.5	No	-	-	-
38	BachTE	26	No	-	-	-
39	TwistedFate	25.3	No	-	-	-

#1Rational Reasoner / IDOLSOTA
80.6
Test· 2023-06-27
IDOL: Indicator-oriented Logic Pre-training for Logical Reasoning Code
#2AMR-LE-Ensemble
80
Test
No paper
#3MERIt(MERIt-deberta-v2-xxlarge )SOTA
79.3
Test· 2022-03-01
MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoning Code
#4MERIt-deberta-v2-xxlarge deberta.v2.xxlarge.path.override_True.norm_1.1.0.w2.A100.cp200.s42
79.3
Test
No paper
#5Knowledge model
79.2
Test
No paper
#6DeBERTa-v2-xxlarge-AMR-LE-Contraposition
77.2
Test
No paper
#7LReasoner ensembleSOTA
76.1
Test· 2021-05-08
Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text Code
#8ELECTRA and ALBERT
71
Test
No paper
#9WWZ
69.7
Test
No paper
#10xlnet-large-uncased [extended data]
69.3
Test
No paper
#11albert_4gpus_bs2_DACE
68.6
Test
No paper
#12Tournament2
66.7
Test
No paper
#13MERIt + Structure Decouple (roberta-large)
65.6
Test
No paper
#14RoBERTa-single
63.5
Test· 2022-05-02
Logiformer: A Two-Branch Graph Transformer Network for Interpretable Logical Reasoning Code
#15MERIT w/ reasoning aware
63.2
Test
No paper
#16ALBERT-XXLarge-V2
62.6
Test
No paper
#17VDGN(RoBERTa single model)
61.7
Test
No paper
#18ro_DA_CE_v100_224
61.7
Test
No paper
#19qmcurtis
60.3
Test
No paper
#20AdaLoGN
60.2
Test
No paper
#21Zhangjie666
60.2
Test
No paper
#22alsace
59.2
Test
No paper
#23RoBERTa-single
58.9
Test· 2021-05-21
Fact-driven Logical Reasoning for Machine Reading Comprehension Code
#24NAACL 2021SOTA
58.2
Test· 2021-03-26
DAGN: Discourse-Aware Graph Network for Logical Reasoning Code
#25Roberta Large GCN(dev: 0.63)
57.7
Test
No paper
#26xlnet_l
57.7
Test
No paper
#27elBERto
57.3
Test
No paper
#285BitClan
56.9
Test
No paper
#29SimChea
56
Test
No paper
#30XLNet-largeSOTA
56
Test· 2020-02-11
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning Code
#31RoBERTa-large
55.6
Test· 2020-02-11
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning Code
#32XLNet-base
50.4
Test· 2020-02-11
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning Code
#33BERT-large+MNLI
50.3
Test
No paper
#34BERT-large
49.8
Test· 2020-02-11
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning Code
#35RoBERTa-base
48.5
Test· 2020-02-11
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning Code
#36BERT-base
47.3
Test· 2020-02-11
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning Code
#37Gariscat
40.5
Test
No paper
#38BachTE
26
Test
No paper
#39TwistedFate
25.3
Test
No paper