Question Answering on HotpotQA

Metric: JOINT-F1 (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	JOINT-F1▼	Extra Data	Paper	Date↕	Code
1	Beam Retrieval	0.775	No	End-to-End Beam Retrieval for Multi-Hop Question...	2023-08-17	Code
2	BigBird-etc	0.736	No	Big Bird: Transformers for Longer Sequences	2020-07-28	Code
3	AISO	0.72	No	Adaptive Information Seeking for Open-Domain Que...	2021-09-14	Code
4	Chain-of-Skills	0.717	No	Chain-of-Skills: A Configurable Model for Open-d...	2023-05-04	Code
5	TPRR	0.708	No	-	-	-
6	HopRetriever + Sp-search	0.706	No	HopRetriever: Retrieve Hops over Wikipedia to An...	2020-12-31	-
7	EBS-Large	0.7	No	-	-	-
8	HopRetriever	0.698	No	-	-	-
9	IRRR+	0.696	No	Answering Open-Domain Questions of Varying Reaso...	2020-10-23	Code
10	EBS-SH	0.689	No	-	-	-
11	IRRR	0.686	No	Answering Open-Domain Questions of Varying Reaso...	2020-10-23	Code
12	HopRetriever-V2	0.678	No	-	-	-
13	AFSGraph-retriever	0.67	No	-	-	-
14	Recursive Dense Retriever	0.666	No	Answering Complex Open-Domain Questions with Mul...	2020-09-27	Code
15	Step-by-Step Retriever	0.662	No	-	-	-
16	DDRQA	0.639	No	Answering Any-hop Open-domain Questions with Ite...	2020-09-16	-
17	HopRetriever-V1	0.639	No	-	-	-
18	DR model large	0.63	No	-	-	-
19	Model name	0.629	No	-	-	-
20	HopAns	0.629	No	-	-	-
21	Anonymous	0.629	No	-	-	-
22	Multi-dimensional-AFSGraph	0.624	No	-	-	-
23	HGN-albert + SemanticRetrievalMRS IR	0.623	No	-	-	-
24	Tree-shaped-cluster	0.617	No	-	-	-
25	AFSgraph	0.617	No	-	-	-
26	Robustly Fine-tuned Graph-based Recurrent Retriever	0.612	No	Learning to Retrieve Reasoning Paths over Wikipe...	2019-11-24	Code
27	AFSgraph model	0.609	No	-	-	-
28	HGN-large + SemanticRetrievalMRS IR	0.607	No	-	-	-
29	RoBERTa-DenseRetriever-Fast	0.602	No	-	-	-
30	DPR-recurrent	0.602	No	-	-	-
31	RoBERTa-DenseRetriever	0.601	No	-	-	-
32	HGN + SemanticRetrievalMRS IR	0.599	No	Hierarchical Graph Network for Multi-hop Questio...	2019-11-09	Code
33	DFGN	0.5982	No	Dynamically Fused Graph Network for Multi-hop Re...	2019-05-16	Code
34	SAFSR model	0.598	No	HotpotQA: A Dataset for Diverse, Explainable Mul...	2018-09-25	Code
35	GraphRR-Fast	0.569	No	-	-	-
36	DR model	0.568	No	-	-	-
37	Quark + SemanticRetrievalMRS IR	0.562	No	A Simple Yet Strong Pipeline for HotpotQA	2020-04-14	-
38	GAR-BERT	0.561	No	-	-	-
39	Graph-based Recurrent Retriever	0.553	No	-	-	-
40	MIR+EPS+BERT	0.548	No	-	-	-
41	GAR	0.53	No	-	-	-
42	Transformer-XH-final	0.513	No	-	-	Code
43	Transformer-XH	0.496	No	-	-	-
44	SemanticRetrievalMRS	0.476	No	Revealing the Importance of Semantic Retrieval f...	2019-09-17	Code
45	DrKIT	0.429	No	-	-	-
46	Entity-centric BERT Pipeline	0.392	No	-	-	-
47	PR-Bert	0.391	No	-	-	-
48	GoldEn Retriever	0.391	No	Answering Complex Open-domain Questions Through ...	2019-10-15	Code
49	SAFSr-Bert	0.37	No	-	-	-
50	Cognitive Graph QA	0.349	No	Cognitive Graph for Multi-Hop Reading Comprehens...	2019-05-14	Code
51	GAR-NOSF	0.334	No	-	-	-
52	IKFGraph	0.304	No	-	-	-
53	AnonymousQ	0.291	No	-	-	-
54	HGN Model-reproduce	0.284	No	-	-	-
55	MUPPET	0.27	No	Multi-Hop Paragraph Retrieval for Open-Domain Qu...	2019-06-15	Code
56	GRN + BERT	0.258	No	-	-	-
57	Entity-centric IR	0.255	No	-	-	-
58	KGNN	0.247	No	Multi-Paragraph Reasoning with Knowledge-enhance...	2019-11-06	-
59	SAQA	0.245	No	-	-	-
60	GRN	0.236	No	-	-	-
61	QFE	0.231	No	Answering while Summarizing: Multi-task Learning...	2019-05-21	-
62	SAFSr_model	0.209	No	-	-	-
63	SuppBERT	0.175	No	-	-	-
64	Baseline Model	0.162	No	HotpotQA: A Dataset for Diverse, Explainable Mul...	2018-09-25	Code
65	tes	0.011	No	-	-	-
66	PromptRank-fewshot-2-demo	0	No	-	-	-
67	graph-recurrent-retriever+roberta-base w. S/R-pretraining	0	No	-	-	-
68	TPReasoner w/o BERT	0	No	-	-	-
69	MultiQA	0	No	-	-	-
70	DecompRC	0	No	-	-	Code
71		0	No	-	-	-
72	Mistral multi hop with very large sources	0	No	-	-	-

#1Beam RetrievalSOTA
0.775
JOINT-F1· 2023-08-17
End-to-End Beam Retrieval for Multi-Hop Question Answering Code
#2BigBird-etcSOTA
0.736
JOINT-F1· 2020-07-28
Big Bird: Transformers for Longer Sequences Code
#3AISO
0.72
JOINT-F1· 2021-09-14
Adaptive Information Seeking for Open-Domain Question Answering Code
#4Chain-of-Skills
0.717
JOINT-F1· 2023-05-04
Chain-of-Skills: A Configurable Model for Open-domain Question Answering Code
#5TPRR
0.708
JOINT-F1
No paper
#6HopRetriever + Sp-search
0.706
JOINT-F1· 2020-12-31
HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions
#7EBS-Large
0.7
JOINT-F1
No paper
#8HopRetriever
0.698
JOINT-F1
No paper
#9IRRR+
0.696
JOINT-F1· 2020-10-23
Answering Open-Domain Questions of Varying Reasoning Steps from Text Code
#10EBS-SH
0.689
JOINT-F1
No paper
#11IRRR
0.686
JOINT-F1· 2020-10-23
Answering Open-Domain Questions of Varying Reasoning Steps from Text Code
#12HopRetriever-V2
0.678
JOINT-F1
No paper
#13AFSGraph-retriever
0.67
JOINT-F1
No paper
#14Recursive Dense Retriever
0.666
JOINT-F1· 2020-09-27
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval Code
#15Step-by-Step Retriever
0.662
JOINT-F1
No paper
#16DDRQA
0.639
JOINT-F1· 2020-09-16
Answering Any-hop Open-domain Questions with Iterative Document Reranking
#17HopRetriever-V1
0.639
JOINT-F1
No paper
#18DR model large
0.63
JOINT-F1
No paper
#19Model name
0.629
JOINT-F1
No paper
#20HopAns
0.629
JOINT-F1
No paper
#21Anonymous
0.629
JOINT-F1
No paper
#22Multi-dimensional-AFSGraph
0.624
JOINT-F1
No paper
#23HGN-albert + SemanticRetrievalMRS IR
0.623
JOINT-F1
No paper
#24Tree-shaped-cluster
0.617
JOINT-F1
No paper
#25AFSgraph
0.617
JOINT-F1
No paper
#26Robustly Fine-tuned Graph-based Recurrent RetrieverSOTA
0.612
JOINT-F1· 2019-11-24
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering Code
#27AFSgraph model
0.609
JOINT-F1
No paper
#28HGN-large + SemanticRetrievalMRS IR
0.607
JOINT-F1
No paper
#29RoBERTa-DenseRetriever-Fast
0.602
JOINT-F1
No paper
#30DPR-recurrent
0.602
JOINT-F1
No paper
#31RoBERTa-DenseRetriever
0.601
JOINT-F1
No paper
#32HGN + SemanticRetrievalMRS IRSOTA
0.599
JOINT-F1· 2019-11-09
Hierarchical Graph Network for Multi-hop Question Answering Code
#33DFGNSOTA
0.5982
JOINT-F1· 2019-05-16
Dynamically Fused Graph Network for Multi-hop Reasoning Code
#34SAFSR modelSOTA
0.598
JOINT-F1· 2018-09-25
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Code
#35GraphRR-Fast
0.569
JOINT-F1
No paper
#36DR model
0.568
JOINT-F1
No paper
#37Quark + SemanticRetrievalMRS IR
0.562
JOINT-F1· 2020-04-14
A Simple Yet Strong Pipeline for HotpotQA
#38GAR-BERT
0.561
JOINT-F1
No paper
#39Graph-based Recurrent Retriever
0.553
JOINT-F1
No paper
#40MIR+EPS+BERT
0.548
JOINT-F1
No paper
#41GAR
0.53
JOINT-F1
No paper
#42Transformer-XH-final
0.513
JOINT-F1
No paperCode
#43Transformer-XH
0.496
JOINT-F1
No paper
#44SemanticRetrievalMRS
0.476
JOINT-F1· 2019-09-17
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale Code
#45DrKIT
0.429
JOINT-F1
No paper
#46Entity-centric BERT Pipeline
0.392
JOINT-F1
No paper
#47PR-Bert
0.391
JOINT-F1
No paper
#48GoldEn Retriever
0.391
JOINT-F1· 2019-10-15
Answering Complex Open-domain Questions Through Iterative Query Generation Code
#49SAFSr-Bert
0.37
JOINT-F1
No paper
#50Cognitive Graph QA
0.349
JOINT-F1· 2019-05-14
Cognitive Graph for Multi-Hop Reading Comprehension at Scale Code
#51GAR-NOSF
0.334
JOINT-F1
No paper
#52IKFGraph
0.304
JOINT-F1
No paper
#53AnonymousQ
0.291
JOINT-F1
No paper
#54HGN Model-reproduce
0.284
JOINT-F1
No paper
#55MUPPET
0.27
JOINT-F1· 2019-06-15
Multi-Hop Paragraph Retrieval for Open-Domain Question Answering Code
#56GRN + BERT
0.258
JOINT-F1
No paper
#57Entity-centric IR
0.255
JOINT-F1
No paper
#58KGNN
0.247
JOINT-F1· 2019-11-06
Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network
#59SAQA
0.245
JOINT-F1
No paper
#60GRN
0.236
JOINT-F1
No paper
#61QFE
0.231
JOINT-F1· 2019-05-21
Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction
#62SAFSr_model
0.209
JOINT-F1
No paper
#63SuppBERT
0.175
JOINT-F1
No paper
#64Baseline Model
0.162
JOINT-F1· 2018-09-25
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Code
#65tes
0.011
JOINT-F1
No paper
#66PromptRank-fewshot-2-demo
0
JOINT-F1
No paper
#67graph-recurrent-retriever+roberta-base w. S/R-pretraining
0
JOINT-F1
No paper
#68TPReasoner w/o BERT
0
JOINT-F1
No paper
#69MultiQA
0
JOINT-F1
No paper
#70DecompRC
0
JOINT-F1
No paperCode
#71
0
JOINT-F1
No paper
#72Mistral multi hop with very large sources
0
JOINT-F1
No paper