Question Answering on HotpotQA

Metric: SUP-F1 (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	SUP-F1▼	Extra Data	Paper	Date↕	Code
1	Beam Retrieval	0.901	No	End-to-End Beam Retrieval for Multi-Hop Question...	2023-08-17	Code
2	BigBird-etc	0.891	No	Big Bird: Transformers for Longer Sequences	2020-07-28	Code
3	AISO	0.86	No	Adaptive Information Seeking for Open-Domain Que...	2021-09-14	Code
4	Chain-of-Skills	0.853	No	Chain-of-Skills: A Configurable Model for Open-d...	2023-05-04	Code
5	TPRR	0.843	No	-	-	-
6	EBS-Large	0.84	No	-	-	-
7	HopRetriever + Sp-search	0.835	No	HopRetriever: Retrieve Hops over Wikipedia to An...	2020-12-31	-
8	IRRR+	0.832	No	Answering Open-Domain Questions of Varying Reaso...	2020-10-23	Code
9	EBS-SH	0.831	No	-	-	-
10	HopRetriever	0.826	No	-	-	-
11	IRRR	0.821	No	Answering Open-Domain Questions of Varying Reaso...	2020-10-23	Code
12	HopRetriever-V2	0.818	No	-	-	-
13	AFSGraph-retriever	0.812	No	-	-	-
14	Recursive Dense Retriever	0.809	No	Answering Complex Open-Domain Questions with Mul...	2020-09-27	Code
15	Step-by-Step Retriever	0.8	No	-	-	-
16	HopRetriever-V1	0.793	No	-	-	-
17	DDRQA	0.789	No	Answering Any-hop Open-domain Questions with Ite...	2020-09-16	-
18	DR model large	0.778	No	-	-	-
19	HGN-albert + SemanticRetrievalMRS IR	0.774	No	-	-	-
20	Model name	0.772	No	-	-	-
21	HopAns	0.772	No	-	-	-
22	Multi-dimensional-AFSGraph	0.772	No	-	-	-
23	Anonymous	0.771	No	-	-	-
24	AFSgraph	0.769	No	-	-	-
25	Tree-shaped-cluster	0.768	No	-	-	-
26	HGN-large + SemanticRetrievalMRS IR	0.768	No	-	-	-
27	Robustly Fine-tuned Graph-based Recurrent Retriever	0.764	No	Learning to Retrieve Reasoning Paths over Wikipe...	2019-11-24	Code
28	HGN + SemanticRetrievalMRS IR	0.764	No	Hierarchical Graph Network for Multi-hop Questio...	2019-11-09	Code
29	AFSgraph model	0.759	No	-	-	-
30	SAFSR model	0.757	No	HotpotQA: A Dataset for Diverse, Explainable Mul...	2018-09-25	Code
31	RoBERTa-DenseRetriever-Fast	0.749	No	-	-	-
32	DPR-recurrent	0.749	No	-	-	-
33	RoBERTa-DenseRetriever	0.748	No	-	-	-
34	GAR-BERT	0.747	No	-	-	-
35	GAR	0.739	No	-	-	-
36	Quark + SemanticRetrievalMRS IR	0.73	No	A Simple Yet Strong Pipeline for HotpotQA	2020-04-14	-
37	Graph-based Recurrent Retriever	0.73	No	-	-	-
38	DR model	0.725	No	-	-	-
39	MIR+EPS+BERT	0.72	No	-	-	-
40	Transformer-XH-final	0.714	No	-	-	Code
41	GraphRR-Fast	0.713	No	-	-	-
42	SemanticRetrievalMRS	0.708	No	Revealing the Importance of Semantic Retrieval f...	2019-09-17	Code
43	Transformer-XH	0.7	No	-	-	-
44	GoldEn Retriever	0.642	No	Answering Complex Open-domain Questions Through ...	2019-10-15	Code
45	DrKIT	0.598	No	-	-	-
46	PR-Bert	0.596	No	-	-	-
47	SAFSr-Bert	0.585	No	-	-	-
48	Cognitive Graph QA	0.577	No	Cognitive Graph for Multi-Hop Reading Comprehens...	2019-05-14	Code
49	Entity-centric BERT Pipeline	0.573	No	-	-	-
50	IKFGraph	0.512	No	-	-	-
51	GRN + BERT	0.497	No	-	-	-
52	HGN Model-reproduce	0.493	No	-	-	-
53	GRN	0.488	No	-	-	-
54	MUPPET	0.473	No	Multi-Hop Paragraph Retrieval for Open-Domain Qu...	2019-06-15	Code
55	KGNN	0.472	No	Multi-Paragraph Reasoning with Knowledge-enhance...	2019-11-06	-
56	SAQA	0.472	No	-	-	-
57	AnonymousQ	0.468	No	-	-	-
58	GAR-NOSF	0.448	No	-	-	-
59	QFE	0.444	No	Answering while Summarizing: Multi-task Learning...	2019-05-21	-
60	Entity-centric IR	0.432	No	-	-	-
61	SAFSr_model	0.406	No	-	-	-
62	SuppBERT	0.4	No	-	-	-
63	Baseline Model	0.377	No	HotpotQA: A Dataset for Diverse, Explainable Mul...	2018-09-25	Code
64	tes	0.078	No	-	-	-
65	PromptRank-fewshot-2-demo	0	No	-	-	-
66	graph-recurrent-retriever+roberta-base w. S/R-pretraining	0	No	-	-	-
67	TPReasoner w/o BERT	0	No	-	-	-
68	MultiQA	0	No	-	-	-
69	DecompRC	0	No	-	-	Code
70		0	No	-	-	-
71	Mistral multi hop with very large sources	0	No	-	-	-

#1Beam RetrievalSOTA
0.901
SUP-F1· 2023-08-17
End-to-End Beam Retrieval for Multi-Hop Question Answering Code
#2BigBird-etcSOTA
0.891
SUP-F1· 2020-07-28
Big Bird: Transformers for Longer Sequences Code
#3AISO
0.86
SUP-F1· 2021-09-14
Adaptive Information Seeking for Open-Domain Question Answering Code
#4Chain-of-Skills
0.853
SUP-F1· 2023-05-04
Chain-of-Skills: A Configurable Model for Open-domain Question Answering Code
#5TPRR
0.843
SUP-F1
No paper
#6EBS-Large
0.84
SUP-F1
No paper
#7HopRetriever + Sp-search
0.835
SUP-F1· 2020-12-31
HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions
#8IRRR+
0.832
SUP-F1· 2020-10-23
Answering Open-Domain Questions of Varying Reasoning Steps from Text Code
#9EBS-SH
0.831
SUP-F1
No paper
#10HopRetriever
0.826
SUP-F1
No paper
#11IRRR
0.821
SUP-F1· 2020-10-23
Answering Open-Domain Questions of Varying Reasoning Steps from Text Code
#12HopRetriever-V2
0.818
SUP-F1
No paper
#13AFSGraph-retriever
0.812
SUP-F1
No paper
#14Recursive Dense Retriever
0.809
SUP-F1· 2020-09-27
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval Code
#15Step-by-Step Retriever
0.8
SUP-F1
No paper
#16HopRetriever-V1
0.793
SUP-F1
No paper
#17DDRQA
0.789
SUP-F1· 2020-09-16
Answering Any-hop Open-domain Questions with Iterative Document Reranking
#18DR model large
0.778
SUP-F1
No paper
#19HGN-albert + SemanticRetrievalMRS IR
0.774
SUP-F1
No paper
#20Model name
0.772
SUP-F1
No paper
#21HopAns
0.772
SUP-F1
No paper
#22Multi-dimensional-AFSGraph
0.772
SUP-F1
No paper
#23Anonymous
0.771
SUP-F1
No paper
#24AFSgraph
0.769
SUP-F1
No paper
#25Tree-shaped-cluster
0.768
SUP-F1
No paper
#26HGN-large + SemanticRetrievalMRS IR
0.768
SUP-F1
No paper
#27Robustly Fine-tuned Graph-based Recurrent Retriever
0.764
SUP-F1· 2019-11-24
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering Code
#28HGN + SemanticRetrievalMRS IRSOTA
0.764
SUP-F1· 2019-11-09
Hierarchical Graph Network for Multi-hop Question Answering Code
#29AFSgraph model
0.759
SUP-F1
No paper
#30SAFSR modelSOTA
0.757
SUP-F1· 2018-09-25
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Code
#31RoBERTa-DenseRetriever-Fast
0.749
SUP-F1
No paper
#32DPR-recurrent
0.749
SUP-F1
No paper
#33RoBERTa-DenseRetriever
0.748
SUP-F1
No paper
#34GAR-BERT
0.747
SUP-F1
No paper
#35GAR
0.739
SUP-F1
No paper
#36Quark + SemanticRetrievalMRS IR
0.73
SUP-F1· 2020-04-14
A Simple Yet Strong Pipeline for HotpotQA
#37Graph-based Recurrent Retriever
0.73
SUP-F1
No paper
#38DR model
0.725
SUP-F1
No paper
#39MIR+EPS+BERT
0.72
SUP-F1
No paper
#40Transformer-XH-final
0.714
SUP-F1
No paperCode
#41GraphRR-Fast
0.713
SUP-F1
No paper
#42SemanticRetrievalMRS
0.708
SUP-F1· 2019-09-17
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale Code
#43Transformer-XH
0.7
SUP-F1
No paper
#44GoldEn Retriever
0.642
SUP-F1· 2019-10-15
Answering Complex Open-domain Questions Through Iterative Query Generation Code
#45DrKIT
0.598
SUP-F1
No paper
#46PR-Bert
0.596
SUP-F1
No paper
#47SAFSr-Bert
0.585
SUP-F1
No paper
#48Cognitive Graph QA
0.577
SUP-F1· 2019-05-14
Cognitive Graph for Multi-Hop Reading Comprehension at Scale Code
#49Entity-centric BERT Pipeline
0.573
SUP-F1
No paper
#50IKFGraph
0.512
SUP-F1
No paper
#51GRN + BERT
0.497
SUP-F1
No paper
#52HGN Model-reproduce
0.493
SUP-F1
No paper
#53GRN
0.488
SUP-F1
No paper
#54MUPPET
0.473
SUP-F1· 2019-06-15
Multi-Hop Paragraph Retrieval for Open-Domain Question Answering Code
#55KGNN
0.472
SUP-F1· 2019-11-06
Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network
#56SAQA
0.472
SUP-F1
No paper
#57AnonymousQ
0.468
SUP-F1
No paper
#58GAR-NOSF
0.448
SUP-F1
No paper
#59QFE
0.444
SUP-F1· 2019-05-21
Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction
#60Entity-centric IR
0.432
SUP-F1
No paper
#61SAFSr_model
0.406
SUP-F1
No paper
#62SuppBERT
0.4
SUP-F1
No paper
#63Baseline Model
0.377
SUP-F1· 2018-09-25
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Code
#64tes
0.078
SUP-F1
No paper
#65PromptRank-fewshot-2-demo
0
SUP-F1
No paper
#66graph-recurrent-retriever+roberta-base w. S/R-pretraining
0
SUP-F1
No paper
#67TPReasoner w/o BERT
0
SUP-F1
No paper
#68MultiQA
0
SUP-F1
No paper
#69DecompRC
0
SUP-F1
No paperCode
#70
0
SUP-F1
No paper
#71Mistral multi hop with very large sources
0
SUP-F1
No paper