Reading Comprehension on RACE

Metric: Accuracy (High) (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	Accuracy (High)▼	Extra Data	Paper	Date↕	Code
1	ALBERTxxlarge+DUMA(ensemble)	92.6	No	DUMA: Reading Comprehension with Transposition T...	2020-01-26	Code
2	Megatron-BERT (ensemble)	90	No	Megatron-LM: Training Multi-Billion Parameter La...	2019-09-17	Code
3	Megatron-BERT	88.6	No	Megatron-LM: Training Multi-Billion Parameter La...	2019-09-17	Code
4	B10-10-10	84.4	No	Funnel-Transformer: Filtering out Sequential Red...	2020-06-05	Code
5	XLNet	84	No	XLNet: Generalized Autoregressive Pretraining fo...	2019-06-19	Code
6	RoBERTa	81.3	No	RoBERTa: A Robustly Optimized BERT Pretraining A...	2019-07-26	Code
7	LLaMA 65B (zero-shot)	51.6	No	LLaMA: Open and Efficient Foundation Language Mo...	2023-02-27	Code
8	PaLM 540B (zero-shot)	49.1	No	PaLM: Scaling Language Modeling with Pathways	2022-04-05	Code
9	LLaMA 33B (zero-shot)	48.3	No	LLaMA: Open and Efficient Foundation Language Mo...	2023-02-27	Code
10	PaLM 62B (zero-shot)	47.5	No	PaLM: Scaling Language Modeling with Pathways	2022-04-05	Code
11	LLaMA 13B (zero-shot)	47.2	No	LLaMA: Open and Efficient Foundation Language Mo...	2023-02-27	Code
12	LLaMA 7B (zero-shot)	46.9	No	LLaMA: Open and Efficient Foundation Language Mo...	2023-02-27	Code
13	GPT-3 175B (zero-shot)	45.5	No	Language Models are Few-Shot Learners	2020-05-28	Code
14	PaLM 8B (zero-shot)	42.3	No	PaLM: Scaling Language Modeling with Pathways	2022-04-05	Code
15	Bloomberg GPT (one-shot)	41.74	No	BloombergGPT: A Large Language Model for Finance	2023-03-30	Code
16	BLOOM 176B (one-shot)	39.14	No	BloombergGPT: A Large Language Model for Finance	2023-03-30	Code
17	OPT 66B (one-shot)	37.02	No	BloombergGPT: A Large Language Model for Finance	2023-03-30	Code
18	GPT-NeoX (one-shot)	34.33	No	BloombergGPT: A Large Language Model for Finance	2023-03-30	Code

#1ALBERTxxlarge+DUMA(ensemble)SOTA
92.6
Accuracy (High)· 2020-01-26
DUMA: Reading Comprehension with Transposition Thinking Code
#2Megatron-BERT (ensemble)SOTA
90
Accuracy (High)· 2019-09-17
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Code
#3Megatron-BERT
88.6
Accuracy (High)· 2019-09-17
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Code
#4B10-10-10
84.4
Accuracy (High)· 2020-06-05
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing Code
#5XLNetSOTA
84
Accuracy (High)· 2019-06-19
XLNet: Generalized Autoregressive Pretraining for Language Understanding Code
#6RoBERTa
81.3
Accuracy (High)· 2019-07-26
RoBERTa: A Robustly Optimized BERT Pretraining Approach Code
#7LLaMA 65B (zero-shot)
51.6
Accuracy (High)· 2023-02-27
LLaMA: Open and Efficient Foundation Language Models Code
#8PaLM 540B (zero-shot)
49.1
Accuracy (High)· 2022-04-05
PaLM: Scaling Language Modeling with Pathways Code
#9LLaMA 33B (zero-shot)
48.3
Accuracy (High)· 2023-02-27
LLaMA: Open and Efficient Foundation Language Models Code
#10PaLM 62B (zero-shot)
47.5
Accuracy (High)· 2022-04-05
PaLM: Scaling Language Modeling with Pathways Code
#11LLaMA 13B (zero-shot)
47.2
Accuracy (High)· 2023-02-27
LLaMA: Open and Efficient Foundation Language Models Code
#12LLaMA 7B (zero-shot)
46.9
Accuracy (High)· 2023-02-27
LLaMA: Open and Efficient Foundation Language Models Code
#13GPT-3 175B (zero-shot)
45.5
Accuracy (High)· 2020-05-28
Language Models are Few-Shot Learners Code
#14PaLM 8B (zero-shot)
42.3
Accuracy (High)· 2022-04-05
PaLM: Scaling Language Modeling with Pathways Code
#15Bloomberg GPT (one-shot)
41.74
Accuracy (High)· 2023-03-30
BloombergGPT: A Large Language Model for Finance Code
#16BLOOM 176B (one-shot)
39.14
Accuracy (High)· 2023-03-30
BloombergGPT: A Large Language Model for Finance Code
#17OPT 66B (one-shot)
37.02
Accuracy (High)· 2023-03-30
BloombergGPT: A Large Language Model for Finance Code
#18GPT-NeoX (one-shot)
34.33
Accuracy (High)· 2023-03-30
BloombergGPT: A Large Language Model for Finance Code