Question Answering on HotpotQA

Metric: JOINT-EM (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	JOINT-EM▼	Extra Data	Paper	Date↕	Code
1	Beam Retrieval	0.505	No	End-to-End Beam Retrieval for Multi-Hop Question...	2023-08-17	Code
2	Chain-of-Skills	0.457	No	Chain-of-Skills: A Configurable Model for Open-d...	2023-05-04	Code
3	AISO	0.449	No	Adaptive Information Seeking for Open-Domain Que...	2021-09-14	Code
4	TPRR	0.444	No	-	-	-
5	HopRetriever + Sp-search	0.432	No	HopRetriever: Retrieve Hops over Wikipedia to An...	2020-12-31	-
6	HopRetriever	0.431	No	-	-	-
7	IRRR+	0.428	No	Answering Open-Domain Questions of Varying Reaso...	2020-10-23	Code
8	IRRR	0.421	No	Answering Open-Domain Questions of Varying Reaso...	2020-10-23	Code
9	EBS-Large	0.42	No	-	-	-
10	Recursive Dense Retriever	0.418	No	Answering Complex Open-Domain Questions with Mul...	2020-09-27	Code
11	AFSGraph-retriever	0.411	No	-	-	-
12	HopRetriever-V2	0.41	No	-	-	-
13	EBS-SH	0.409	No	-	-	-
14	Step-by-Step Retriever	0.404	No	-	-	-
15	HopRetriever-V1	0.38	No	-	-	-
16	Anonymous	0.38	No	-	-	-
17	HGN-albert + SemanticRetrievalMRS IR	0.379	No	-	-	-
18	HGN-large + SemanticRetrievalMRS IR	0.372	No	-	-	-
19	Model name	0.368	No	-	-	-
20	HopAns	0.368	No	-	-	-
21	Multi-dimensional-AFSGraph	0.362	No	-	-	-
22	DDRQA	0.36	No	Answering Any-hop Open-domain Questions with Ite...	2020-09-16	-
23	Tree-shaped-cluster	0.359	No	-	-	-
24	AFSgraph	0.359	No	-	-	-
25	HGN + SemanticRetrievalMRS IR	0.356	No	Hierarchical Graph Network for Multi-hop Questio...	2019-11-09	Code
26	DR model large	0.354	No	-	-	-
27	Robustly Fine-tuned Graph-based Recurrent Retriever	0.354	No	Learning to Retrieve Reasoning Paths over Wikipe...	2019-11-24	Code
28	AFSgraph model	0.35	No	-	-	-
29	RoBERTa-DenseRetriever-Fast	0.345	No	-	-	-
30	DPR-recurrent	0.345	No	-	-	-
31	RoBERTa-DenseRetriever	0.345	No	-	-	-
32	SAFSR model	0.345	No	HotpotQA: A Dataset for Diverse, Explainable Mul...	2018-09-25	Code
33	GAR-BERT	0.33	No	-	-	-
34	Quark + SemanticRetrievalMRS IR	0.329	No	A Simple Yet Strong Pipeline for HotpotQA	2020-04-14	-
35	MIR+EPS+BERT	0.312	No	-	-	-
36	GraphRR-Fast	0.31	No	-	-	-
37	GAR	0.306	No	-	-	-
38	DR model	0.293	No	-	-	-
39	Graph-based Recurrent Retriever	0.292	No	-	-	-
40	Transformer-XH	0.271	No	-	-	-
41	Transformer-XH-final	0.261	No	-	-	Code
42	SemanticRetrievalMRS	0.251	No	Revealing the Importance of Semantic Retrieval f...	2019-09-17	Code
43	DrKIT	0.247	No	-	-	-
44	GoldEn Retriever	0.18	No	Answering Complex Open-domain Questions Through ...	2019-10-15	Code
45	Entity-centric BERT Pipeline	0.17	No	-	-	-
46	PR-Bert	0.145	No	-	-	-
47	SAFSr-Bert	0.133	No	-	-	-
48	Cognitive Graph QA	0.124	No	Cognitive Graph for Multi-Hop Reading Comprehens...	2019-05-14	Code
49	IKFGraph	0.115	No	-	-	-
50	AnonymousQ	0.115	No	-	-	-
51	HGN Model-reproduce	0.11	No	-	-	-
52	MUPPET	0.109	No	Multi-Hop Paragraph Retrieval for Open-Domain Qu...	2019-06-15	Code
53	QFE	0.087	No	Answering while Summarizing: Multi-task Learning...	2019-05-21	-
54	SAQA	0.086	No	-	-	-
55	GRN + BERT	0.083	No	-	-	-
56	GRN	0.074	No	-	-	-
57	KGNN	0.07	No	Multi-Paragraph Reasoning with Knowledge-enhance...	2019-11-06	-
58	GAR-NOSF	0.049	No	-	-	-
59	SAFSr_model	0.041	No	-	-	-
60	SuppBERT	0.033	No	-	-	-
61	Baseline Model	0.019	No	HotpotQA: A Dataset for Diverse, Explainable Mul...	2018-09-25	Code
62	Entity-centric IR	0	No	-	-	-
63	tes	0	No	-	-	-
64	PromptRank-fewshot-2-demo	0	No	-	-	-
65	graph-recurrent-retriever+roberta-base w. S/R-pretraining	0	No	-	-	-
66	TPReasoner w/o BERT	0	No	-	-	-
67	MultiQA	0	No	-	-	-
68	DecompRC	0	No	-	-	Code
69		0	No	-	-	-
70	Mistral multi hop with very large sources	0	No	-	-	-

#1Beam RetrievalSOTA
0.505
JOINT-EM· 2023-08-17
End-to-End Beam Retrieval for Multi-Hop Question Answering Code
#2Chain-of-SkillsSOTA
0.457
JOINT-EM· 2023-05-04
Chain-of-Skills: A Configurable Model for Open-domain Question Answering Code
#3AISOSOTA
0.449
JOINT-EM· 2021-09-14
Adaptive Information Seeking for Open-Domain Question Answering Code
#4TPRR
0.444
JOINT-EM
No paper
#5HopRetriever + Sp-searchSOTA
0.432
JOINT-EM· 2020-12-31
HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions
#6HopRetriever
0.431
JOINT-EM
No paper
#7IRRR+SOTA
0.428
JOINT-EM· 2020-10-23
Answering Open-Domain Questions of Varying Reasoning Steps from Text Code
#8IRRR
0.421
JOINT-EM· 2020-10-23
Answering Open-Domain Questions of Varying Reasoning Steps from Text Code
#9EBS-Large
0.42
JOINT-EM
No paper
#10Recursive Dense RetrieverSOTA
0.418
JOINT-EM· 2020-09-27
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval Code
#11AFSGraph-retriever
0.411
JOINT-EM
No paper
#12HopRetriever-V2
0.41
JOINT-EM
No paper
#13EBS-SH
0.409
JOINT-EM
No paper
#14Step-by-Step Retriever
0.404
JOINT-EM
No paper
#15HopRetriever-V1
0.38
JOINT-EM
No paper
#16Anonymous
0.38
JOINT-EM
No paper
#17HGN-albert + SemanticRetrievalMRS IR
0.379
JOINT-EM
No paper
#18HGN-large + SemanticRetrievalMRS IR
0.372
JOINT-EM
No paper
#19Model name
0.368
JOINT-EM
No paper
#20HopAns
0.368
JOINT-EM
No paper
#21Multi-dimensional-AFSGraph
0.362
JOINT-EM
No paper
#22DDRQASOTA
0.36
JOINT-EM· 2020-09-16
Answering Any-hop Open-domain Questions with Iterative Document Reranking
#23Tree-shaped-cluster
0.359
JOINT-EM
No paper
#24AFSgraph
0.359
JOINT-EM
No paper
#25HGN + SemanticRetrievalMRS IRSOTA
0.356
JOINT-EM· 2019-11-09
Hierarchical Graph Network for Multi-hop Question Answering Code
#26DR model large
0.354
JOINT-EM
No paper
#27Robustly Fine-tuned Graph-based Recurrent Retriever
0.354
JOINT-EM· 2019-11-24
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering Code
#28AFSgraph model
0.35
JOINT-EM
No paper
#29RoBERTa-DenseRetriever-Fast
0.345
JOINT-EM
No paper
#30DPR-recurrent
0.345
JOINT-EM
No paper
#31RoBERTa-DenseRetriever
0.345
JOINT-EM
No paper
#32SAFSR modelSOTA
0.345
JOINT-EM· 2018-09-25
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Code
#33GAR-BERT
0.33
JOINT-EM
No paper
#34Quark + SemanticRetrievalMRS IR
0.329
JOINT-EM· 2020-04-14
A Simple Yet Strong Pipeline for HotpotQA
#35MIR+EPS+BERT
0.312
JOINT-EM
No paper
#36GraphRR-Fast
0.31
JOINT-EM
No paper
#37GAR
0.306
JOINT-EM
No paper
#38DR model
0.293
JOINT-EM
No paper
#39Graph-based Recurrent Retriever
0.292
JOINT-EM
No paper
#40Transformer-XH
0.271
JOINT-EM
No paper
#41Transformer-XH-final
0.261
JOINT-EM
No paperCode
#42SemanticRetrievalMRS
0.251
JOINT-EM· 2019-09-17
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale Code
#43DrKIT
0.247
JOINT-EM
No paper
#44GoldEn Retriever
0.18
JOINT-EM· 2019-10-15
Answering Complex Open-domain Questions Through Iterative Query Generation Code
#45Entity-centric BERT Pipeline
0.17
JOINT-EM
No paper
#46PR-Bert
0.145
JOINT-EM
No paper
#47SAFSr-Bert
0.133
JOINT-EM
No paper
#48Cognitive Graph QA
0.124
JOINT-EM· 2019-05-14
Cognitive Graph for Multi-Hop Reading Comprehension at Scale Code
#49IKFGraph
0.115
JOINT-EM
No paper
#50AnonymousQ
0.115
JOINT-EM
No paper
#51HGN Model-reproduce
0.11
JOINT-EM
No paper
#52MUPPET
0.109
JOINT-EM· 2019-06-15
Multi-Hop Paragraph Retrieval for Open-Domain Question Answering Code
#53QFE
0.087
JOINT-EM· 2019-05-21
Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction
#54SAQA
0.086
JOINT-EM
No paper
#55GRN + BERT
0.083
JOINT-EM
No paper
#56GRN
0.074
JOINT-EM
No paper
#57KGNN
0.07
JOINT-EM· 2019-11-06
Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network
#58GAR-NOSF
0.049
JOINT-EM
No paper
#59SAFSr_model
0.041
JOINT-EM
No paper
#60SuppBERT
0.033
JOINT-EM
No paper
#61Baseline Model
0.019
JOINT-EM· 2018-09-25
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Code
#62Entity-centric IR
0
JOINT-EM
No paper
#63tes
0
JOINT-EM
No paper
#64PromptRank-fewshot-2-demo
0
JOINT-EM
No paper
#65graph-recurrent-retriever+roberta-base w. S/R-pretraining
0
JOINT-EM
No paper
#66TPReasoner w/o BERT
0
JOINT-EM
No paper
#67MultiQA
0
JOINT-EM
No paper
#68DecompRC
0
JOINT-EM
No paperCode
#69
0
JOINT-EM
No paper
#70Mistral multi hop with very large sources
0
JOINT-EM
No paper