Question Answering on HotpotQA

Metric: SUP-EM (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	SUP-EM▼	Extra Data	Paper	Date↕	Code
1	Beam Retrieval	0.663	No	End-to-End Beam Retrieval for Multi-Hop Question...	2023-08-17	Code
2	Chain-of-Skills	0.613	No	Chain-of-Skills: A Configurable Model for Open-d...	2023-05-04	Code
3	AISO	0.612	No	Adaptive Information Seeking for Open-Domain Que...	2021-09-14	Code
4	TPRR	0.594	No	-	-	-
5	Recursive Dense Retriever	0.575	No	Answering Complex Open-Domain Questions with Mul...	2020-09-27	Code
6	HopRetriever + Sp-search	0.574	No	HopRetriever: Retrieve Hops over Wikipedia to An...	2020-12-31	-
7	EBS-Large	0.573	No	-	-	-
8	HopRetriever	0.572	No	-	-	-
9	IRRR+	0.569	No	Answering Open-Domain Questions of Varying Reaso...	2020-10-23	Code
10	HopRetriever-V2	0.561	No	-	-	-
11	EBS-SH	0.559	No	-	-	-
12	IRRR	0.559	No	Answering Open-Domain Questions of Varying Reaso...	2020-10-23	Code
13	AFSGraph-retriever	0.557	No	-	-	-
14	Step-by-Step Retriever	0.546	No	-	-	-
15	HopRetriever-V1	0.531	No	-	-	-
16	Anonymous	0.52	No	-	-	-
17	DDRQA	0.51	No	Answering Any-hop Open-domain Questions with Ite...	2020-09-16	-
18	HGN-albert + SemanticRetrievalMRS IR	0.51	No	-	-	-
19	HGN-large + SemanticRetrievalMRS IR	0.51	No	-	-	-
20	Multi-dimensional-AFSGraph	0.503	No	-	-	-
21	Model name	0.5	No	-	-	-
22	HopAns	0.5	No	-	-	-
23	AFSgraph	0.5	No	-	-	-
24	HGN + SemanticRetrievalMRS IR	0.5	No	Hierarchical Graph Network for Multi-hop Questio...	2019-11-09	Code
25	DR model large	0.499	No	-	-	-
26	Tree-shaped-cluster	0.499	No	-	-	-
27	Robustly Fine-tuned Graph-based Recurrent Retriever	0.491	No	Learning to Retrieve Reasoning Paths over Wikipe...	2019-11-24	Code
28	GAR-BERT	0.49	No	-	-	-
29	AFSgraph model	0.485	No	-	-	-
30	GAR	0.483	No	-	-	-
31	RoBERTa-DenseRetriever-Fast	0.48	No	-	-	-
32	DPR-recurrent	0.48	No	-	-	-
33	SAFSR model	0.48	No	HotpotQA: A Dataset for Diverse, Explainable Mul...	2018-09-25	Code
34	RoBERTa-DenseRetriever	0.479	No	-	-	-
35	Quark + SemanticRetrievalMRS IR	0.456	No	A Simple Yet Strong Pipeline for HotpotQA	2020-04-14	-
36	Graph-based Recurrent Retriever	0.441	No	-	-	-
37	GraphRR-Fast	0.429	No	-	-	-
38	MIR+EPS+BERT	0.428	No	-	-	-
39	Transformer-XH	0.417	No	-	-	-
40	DR model	0.416	No	-	-	-
41	Transformer-XH-final	0.409	No	-	-	Code
42	SemanticRetrievalMRS	0.387	No	Revealing the Importance of Semantic Retrieval f...	2019-09-17	Code
43	DrKIT	0.371	No	-	-	-
44	GoldEn Retriever	0.307	No	Answering Complex Open-domain Questions Through ...	2019-10-15	Code
45	Entity-centric BERT Pipeline	0.263	No	-	-	-
46	SAFSr-Bert	0.242	No	-	-	-
47	Cognitive Graph QA	0.228	No	Cognitive Graph for Multi-Hop Reading Comprehens...	2019-05-14	Code
48	PR-Bert	0.219	No	-	-	-
49	MUPPET	0.167	No	Multi-Hop Paragraph Retrieval for Open-Domain Qu...	2019-06-15	Code
50	IKFGraph	0.16	No	-	-	-
51	HGN Model-reproduce	0.156	No	-	-	-
52	AnonymousQ	0.153	No	-	-	-
53	SAQA	0.147	No	-	-	-
54	QFE	0.142	No	Answering while Summarizing: Multi-task Learning...	2019-05-21	-
55	GRN + BERT	0.132	No	-	-	-
56	KGNN	0.127	No	Multi-Paragraph Reasoning with Knowledge-enhance...	2019-11-06	-
57	GRN	0.122	No	-	-	-
58	SAFSr_model	0.08	No	-	-	-
59	GAR-NOSF	0.076	No	-	-	-
60	SuppBERT	0.056	No	-	-	-
61	Baseline Model	0.039	No	HotpotQA: A Dataset for Diverse, Explainable Mul...	2018-09-25	Code
62	Entity-centric IR	0.001	No	-	-	-
63	tes	0	No	-	-	-
64	PromptRank-fewshot-2-demo	0	No	-	-	-
65	graph-recurrent-retriever+roberta-base w. S/R-pretraining	0	No	-	-	-
66	TPReasoner w/o BERT	0	No	-	-	-
67	MultiQA	0	No	-	-	-
68	DecompRC	0	No	-	-	Code
69		0	No	-	-	-
70	Mistral multi hop with very large sources	0	No	-	-	-

#1Beam RetrievalSOTA
0.663
SUP-EM· 2023-08-17
End-to-End Beam Retrieval for Multi-Hop Question Answering Code
#2Chain-of-SkillsSOTA
0.613
SUP-EM· 2023-05-04
Chain-of-Skills: A Configurable Model for Open-domain Question Answering Code
#3AISOSOTA
0.612
SUP-EM· 2021-09-14
Adaptive Information Seeking for Open-Domain Question Answering Code
#4TPRR
0.594
SUP-EM
No paper
#5Recursive Dense RetrieverSOTA
0.575
SUP-EM· 2020-09-27
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval Code
#6HopRetriever + Sp-search
0.574
SUP-EM· 2020-12-31
HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions
#7EBS-Large
0.573
SUP-EM
No paper
#8HopRetriever
0.572
SUP-EM
No paper
#9IRRR+
0.569
SUP-EM· 2020-10-23
Answering Open-Domain Questions of Varying Reasoning Steps from Text Code
#10HopRetriever-V2
0.561
SUP-EM
No paper
#11EBS-SH
0.559
SUP-EM
No paper
#12IRRR
0.559
SUP-EM· 2020-10-23
Answering Open-Domain Questions of Varying Reasoning Steps from Text Code
#13AFSGraph-retriever
0.557
SUP-EM
No paper
#14Step-by-Step Retriever
0.546
SUP-EM
No paper
#15HopRetriever-V1
0.531
SUP-EM
No paper
#16Anonymous
0.52
SUP-EM
No paper
#17DDRQASOTA
0.51
SUP-EM· 2020-09-16
Answering Any-hop Open-domain Questions with Iterative Document Reranking
#18HGN-albert + SemanticRetrievalMRS IR
0.51
SUP-EM
No paper
#19HGN-large + SemanticRetrievalMRS IR
0.51
SUP-EM
No paper
#20Multi-dimensional-AFSGraph
0.503
SUP-EM
No paper
#21Model name
0.5
SUP-EM
No paper
#22HopAns
0.5
SUP-EM
No paper
#23AFSgraph
0.5
SUP-EM
No paper
#24HGN + SemanticRetrievalMRS IRSOTA
0.5
SUP-EM· 2019-11-09
Hierarchical Graph Network for Multi-hop Question Answering Code
#25DR model large
0.499
SUP-EM
No paper
#26Tree-shaped-cluster
0.499
SUP-EM
No paper
#27Robustly Fine-tuned Graph-based Recurrent Retriever
0.491
SUP-EM· 2019-11-24
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering Code
#28GAR-BERT
0.49
SUP-EM
No paper
#29AFSgraph model
0.485
SUP-EM
No paper
#30GAR
0.483
SUP-EM
No paper
#31RoBERTa-DenseRetriever-Fast
0.48
SUP-EM
No paper
#32DPR-recurrent
0.48
SUP-EM
No paper
#33SAFSR modelSOTA
0.48
SUP-EM· 2018-09-25
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Code
#34RoBERTa-DenseRetriever
0.479
SUP-EM
No paper
#35Quark + SemanticRetrievalMRS IR
0.456
SUP-EM· 2020-04-14
A Simple Yet Strong Pipeline for HotpotQA
#36Graph-based Recurrent Retriever
0.441
SUP-EM
No paper
#37GraphRR-Fast
0.429
SUP-EM
No paper
#38MIR+EPS+BERT
0.428
SUP-EM
No paper
#39Transformer-XH
0.417
SUP-EM
No paper
#40DR model
0.416
SUP-EM
No paper
#41Transformer-XH-final
0.409
SUP-EM
No paperCode
#42SemanticRetrievalMRS
0.387
SUP-EM· 2019-09-17
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale Code
#43DrKIT
0.371
SUP-EM
No paper
#44GoldEn Retriever
0.307
SUP-EM· 2019-10-15
Answering Complex Open-domain Questions Through Iterative Query Generation Code
#45Entity-centric BERT Pipeline
0.263
SUP-EM
No paper
#46SAFSr-Bert
0.242
SUP-EM
No paper
#47Cognitive Graph QA
0.228
SUP-EM· 2019-05-14
Cognitive Graph for Multi-Hop Reading Comprehension at Scale Code
#48PR-Bert
0.219
SUP-EM
No paper
#49MUPPET
0.167
SUP-EM· 2019-06-15
Multi-Hop Paragraph Retrieval for Open-Domain Question Answering Code
#50IKFGraph
0.16
SUP-EM
No paper
#51HGN Model-reproduce
0.156
SUP-EM
No paper
#52AnonymousQ
0.153
SUP-EM
No paper
#53SAQA
0.147
SUP-EM
No paper
#54QFE
0.142
SUP-EM· 2019-05-21
Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction
#55GRN + BERT
0.132
SUP-EM
No paper
#56KGNN
0.127
SUP-EM· 2019-11-06
Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network
#57GRN
0.122
SUP-EM
No paper
#58SAFSr_model
0.08
SUP-EM
No paper
#59GAR-NOSF
0.076
SUP-EM
No paper
#60SuppBERT
0.056
SUP-EM
No paper
#61Baseline Model
0.039
SUP-EM· 2018-09-25
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Code
#62Entity-centric IR
0.001
SUP-EM
No paper
#63tes
0
SUP-EM
No paper
#64PromptRank-fewshot-2-demo
0
SUP-EM
No paper
#65graph-recurrent-retriever+roberta-base w. S/R-pretraining
0
SUP-EM
No paper
#66TPReasoner w/o BERT
0
SUP-EM
No paper
#67MultiQA
0
SUP-EM
No paper
#68DecompRC
0
SUP-EM
No paperCode
#69
0
SUP-EM
No paper
#70Mistral multi hop with very large sources
0
SUP-EM
No paper