Reading Comprehension on RACE

Metric: Accuracy (Middle) (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	Accuracy (Middle)▼	Extra Data	Paper	Date↕	Code
1	Megatron-BERT (ensemble)	93.1	No	Megatron-LM: Training Multi-Billion Parameter La...	2019-09-17	Code
2	Megatron-BERT	91.8	No	Megatron-LM: Training Multi-Billion Parameter La...	2019-09-17	Code
3	B10-10-10	88.8	No	Funnel-Transformer: Filtering out Sequential Red...	2020-06-05	Code
4	ALBERTxxlarge+DUMA(ensemble)	88.7	No	DUMA: Reading Comprehension with Transposition T...	2020-01-26	Code
5	XLNet	88.6	No	XLNet: Generalized Autoregressive Pretraining fo...	2019-06-19	Code
6	RoBERTa	86.5	No	RoBERTa: A Robustly Optimized BERT Pretraining A...	2019-07-26	Code
7	PaLM 540B (zero-shot)	68.1	No	PaLM: Scaling Language Modeling with Pathways	2022-04-05	Code
8	LLaMA 65B (zero-shot)	67.9	No	LLaMA: Open and Efficient Foundation Language Mo...	2023-02-27	Code
9	PaLM 62B (zero-shot)	64.3	No	PaLM: Scaling Language Modeling with Pathways	2022-04-05	Code
10	LLaMA 33B (zero-shot)	64.1	No	LLaMA: Open and Efficient Foundation Language Mo...	2023-02-27	Code
11	LLaMA 13B (zero-shot)	61.6	No	LLaMA: Open and Efficient Foundation Language Mo...	2023-02-27	Code
12	LLaMA 7B (zero-shot)	61.1	No	LLaMA: Open and Efficient Foundation Language Mo...	2023-02-27	Code
13	GPT-3 175B (0-shot)	58.4	No	Language Models are Few-Shot Learners	2020-05-28	Code
14	PaLM 8B (zero-shot)	57.9	No	PaLM: Scaling Language Modeling with Pathways	2022-04-05	Code
15	Bloomberg GPT (one-shot)	54.32	No	BloombergGPT: A Large Language Model for Finance	2023-03-30	Code
16	BLOOM 176B (one-shot)	52.3	No	BloombergGPT: A Large Language Model for Finance	2023-03-30	Code
17	OPT 66B (one-shot)	47.42	No	BloombergGPT: A Large Language Model for Finance	2023-03-30	Code
18	GPT-NeoX (one-shot)	41.23	No	BloombergGPT: A Large Language Model for Finance	2023-03-30	Code

#1Megatron-BERT (ensemble)SOTA
93.1
Accuracy (Middle)· 2019-09-17
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Code
#2Megatron-BERT
91.8
Accuracy (Middle)· 2019-09-17
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Code
#3B10-10-10
88.8
Accuracy (Middle)· 2020-06-05
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing Code
#4ALBERTxxlarge+DUMA(ensemble)
88.7
Accuracy (Middle)· 2020-01-26
DUMA: Reading Comprehension with Transposition Thinking Code
#5XLNetSOTA
88.6
Accuracy (Middle)· 2019-06-19
XLNet: Generalized Autoregressive Pretraining for Language Understanding Code
#6RoBERTa
86.5
Accuracy (Middle)· 2019-07-26
RoBERTa: A Robustly Optimized BERT Pretraining Approach Code
#7PaLM 540B (zero-shot)
68.1
Accuracy (Middle)· 2022-04-05
PaLM: Scaling Language Modeling with Pathways Code
#8LLaMA 65B (zero-shot)
67.9
Accuracy (Middle)· 2023-02-27
LLaMA: Open and Efficient Foundation Language Models Code
#9PaLM 62B (zero-shot)
64.3
Accuracy (Middle)· 2022-04-05
PaLM: Scaling Language Modeling with Pathways Code
#10LLaMA 33B (zero-shot)
64.1
Accuracy (Middle)· 2023-02-27
LLaMA: Open and Efficient Foundation Language Models Code
#11LLaMA 13B (zero-shot)
61.6
Accuracy (Middle)· 2023-02-27
LLaMA: Open and Efficient Foundation Language Models Code
#12LLaMA 7B (zero-shot)
61.1
Accuracy (Middle)· 2023-02-27
LLaMA: Open and Efficient Foundation Language Models Code
#13GPT-3 175B (0-shot)
58.4
Accuracy (Middle)· 2020-05-28
Language Models are Few-Shot Learners Code
#14PaLM 8B (zero-shot)
57.9
Accuracy (Middle)· 2022-04-05
PaLM: Scaling Language Modeling with Pathways Code
#15Bloomberg GPT (one-shot)
54.32
Accuracy (Middle)· 2023-03-30
BloombergGPT: A Large Language Model for Finance Code
#16BLOOM 176B (one-shot)
52.3
Accuracy (Middle)· 2023-03-30
BloombergGPT: A Large Language Model for Finance Code
#17OPT 66B (one-shot)
47.42
Accuracy (Middle)· 2023-03-30
BloombergGPT: A Large Language Model for Finance Code
#18GPT-NeoX (one-shot)
41.23
Accuracy (Middle)· 2023-03-30
BloombergGPT: A Large Language Model for Finance Code