Tasks
SotA
Datasets
Papers
Methods
Submit
About
SotA
/
Natural Language Processing
/
Question Answering
/
HotpotQA
Question Answering on HotpotQA
Metric: SUP-EM (higher is better)
Leaderboard
Dataset
Loading chart...
Results
Submit a result
Export CSV
Sort:
SUP-EM (best first)
SUP-EM (worst first)
Date (newest first)
Date (oldest first)
Model name (A→Z)
#
Model
↕
SUP-EM
▼
Extra Data
Paper
Date
↕
Code
1
Beam Retrieval
0.663
No
End-to-End Beam Retrieval for Multi-Hop Question...
2023-08-17
Code
2
Chain-of-Skills
0.613
No
Chain-of-Skills: A Configurable Model for Open-d...
2023-05-04
Code
3
AISO
0.612
No
Adaptive Information Seeking for Open-Domain Que...
2021-09-14
Code
4
TPRR
0.594
No
-
-
-
5
Recursive Dense Retriever
0.575
No
Answering Complex Open-Domain Questions with Mul...
2020-09-27
Code
6
HopRetriever + Sp-search
0.574
No
HopRetriever: Retrieve Hops over Wikipedia to An...
2020-12-31
-
7
EBS-Large
0.573
No
-
-
-
8
HopRetriever
0.572
No
-
-
-
9
IRRR+
0.569
No
Answering Open-Domain Questions of Varying Reaso...
2020-10-23
Code
10
HopRetriever-V2
0.561
No
-
-
-
11
EBS-SH
0.559
No
-
-
-
12
IRRR
0.559
No
Answering Open-Domain Questions of Varying Reaso...
2020-10-23
Code
13
AFSGraph-retriever
0.557
No
-
-
-
14
Step-by-Step Retriever
0.546
No
-
-
-
15
HopRetriever-V1
0.531
No
-
-
-
16
Anonymous
0.52
No
-
-
-
17
DDRQA
0.51
No
Answering Any-hop Open-domain Questions with Ite...
2020-09-16
-
18
HGN-albert + SemanticRetrievalMRS IR
0.51
No
-
-
-
19
HGN-large + SemanticRetrievalMRS IR
0.51
No
-
-
-
20
Multi-dimensional-AFSGraph
0.503
No
-
-
-
21
Model name
0.5
No
-
-
-
22
HopAns
0.5
No
-
-
-
23
AFSgraph
0.5
No
-
-
-
24
HGN + SemanticRetrievalMRS IR
0.5
No
Hierarchical Graph Network for Multi-hop Questio...
2019-11-09
Code
25
DR model large
0.499
No
-
-
-
26
Tree-shaped-cluster
0.499
No
-
-
-
27
Robustly Fine-tuned Graph-based Recurrent Retriever
0.491
No
Learning to Retrieve Reasoning Paths over Wikipe...
2019-11-24
Code
28
GAR-BERT
0.49
No
-
-
-
29
AFSgraph model
0.485
No
-
-
-
30
GAR
0.483
No
-
-
-
31
RoBERTa-DenseRetriever-Fast
0.48
No
-
-
-
32
DPR-recurrent
0.48
No
-
-
-
33
SAFSR model
0.48
No
HotpotQA: A Dataset for Diverse, Explainable Mul...
2018-09-25
Code
34
RoBERTa-DenseRetriever
0.479
No
-
-
-
35
Quark + SemanticRetrievalMRS IR
0.456
No
A Simple Yet Strong Pipeline for HotpotQA
2020-04-14
-
36
Graph-based Recurrent Retriever
0.441
No
-
-
-
37
GraphRR-Fast
0.429
No
-
-
-
38
MIR+EPS+BERT
0.428
No
-
-
-
39
Transformer-XH
0.417
No
-
-
-
40
DR model
0.416
No
-
-
-
41
Transformer-XH-final
0.409
No
-
-
Code
42
SemanticRetrievalMRS
0.387
No
Revealing the Importance of Semantic Retrieval f...
2019-09-17
Code
43
DrKIT
0.371
No
-
-
-
44
GoldEn Retriever
0.307
No
Answering Complex Open-domain Questions Through ...
2019-10-15
Code
45
Entity-centric BERT Pipeline
0.263
No
-
-
-
46
SAFSr-Bert
0.242
No
-
-
-
47
Cognitive Graph QA
0.228
No
Cognitive Graph for Multi-Hop Reading Comprehens...
2019-05-14
Code
48
PR-Bert
0.219
No
-
-
-
49
MUPPET
0.167
No
Multi-Hop Paragraph Retrieval for Open-Domain Qu...
2019-06-15
Code
50
IKFGraph
0.16
No
-
-
-
51
HGN Model-reproduce
0.156
No
-
-
-
52
AnonymousQ
0.153
No
-
-
-
53
SAQA
0.147
No
-
-
-
54
QFE
0.142
No
Answering while Summarizing: Multi-task Learning...
2019-05-21
-
55
GRN + BERT
0.132
No
-
-
-
56
KGNN
0.127
No
Multi-Paragraph Reasoning with Knowledge-enhance...
2019-11-06
-
57
GRN
0.122
No
-
-
-
58
SAFSr_model
0.08
No
-
-
-
59
GAR-NOSF
0.076
No
-
-
-
60
SuppBERT
0.056
No
-
-
-
61
Baseline Model
0.039
No
HotpotQA: A Dataset for Diverse, Explainable Mul...
2018-09-25
Code
62
Entity-centric IR
0.001
No
-
-
-
63
tes
0
No
-
-
-
64
PromptRank-fewshot-2-demo
0
No
-
-
-
65
graph-recurrent-retriever+roberta-base w. S/R-pretraining
0
No
-
-
-
66
TPReasoner w/o BERT
0
No
-
-
-
67
MultiQA
0
No
-
-
-
68
DecompRC
0
No
-
-
Code
69
0
No
-
-
-
70
Mistral multi hop with very large sources
0
No
-
-
-
#1
Beam Retrieval
SOTA
0.663
SUP-EM
· 2023-08-17
End-to-End Beam Retrieval for Multi-Hop Question Answering
Code
#2
Chain-of-Skills
SOTA
0.613
SUP-EM
· 2023-05-04
Chain-of-Skills: A Configurable Model for Open-domain Question Answering
Code
#3
AISO
SOTA
0.612
SUP-EM
· 2021-09-14
Adaptive Information Seeking for Open-Domain Question Answering
Code
#4
TPRR
0.594
SUP-EM
No paper
#5
Recursive Dense Retriever
SOTA
0.575
SUP-EM
· 2020-09-27
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval
Code
#6
HopRetriever + Sp-search
0.574
SUP-EM
· 2020-12-31
HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions
#7
EBS-Large
0.573
SUP-EM
No paper
#8
HopRetriever
0.572
SUP-EM
No paper
#9
IRRR+
0.569
SUP-EM
· 2020-10-23
Answering Open-Domain Questions of Varying Reasoning Steps from Text
Code
#10
HopRetriever-V2
0.561
SUP-EM
No paper
#11
EBS-SH
0.559
SUP-EM
No paper
#12
IRRR
0.559
SUP-EM
· 2020-10-23
Answering Open-Domain Questions of Varying Reasoning Steps from Text
Code
#13
AFSGraph-retriever
0.557
SUP-EM
No paper
#14
Step-by-Step Retriever
0.546
SUP-EM
No paper
#15
HopRetriever-V1
0.531
SUP-EM
No paper
#16
Anonymous
0.52
SUP-EM
No paper
#17
DDRQA
SOTA
0.51
SUP-EM
· 2020-09-16
Answering Any-hop Open-domain Questions with Iterative Document Reranking
#18
HGN-albert + SemanticRetrievalMRS IR
0.51
SUP-EM
No paper
#19
HGN-large + SemanticRetrievalMRS IR
0.51
SUP-EM
No paper
#20
Multi-dimensional-AFSGraph
0.503
SUP-EM
No paper
#21
Model name
0.5
SUP-EM
No paper
#22
HopAns
0.5
SUP-EM
No paper
#23
AFSgraph
0.5
SUP-EM
No paper
#24
HGN + SemanticRetrievalMRS IR
SOTA
0.5
SUP-EM
· 2019-11-09
Hierarchical Graph Network for Multi-hop Question Answering
Code
#25
DR model large
0.499
SUP-EM
No paper
#26
Tree-shaped-cluster
0.499
SUP-EM
No paper
#27
Robustly Fine-tuned Graph-based Recurrent Retriever
0.491
SUP-EM
· 2019-11-24
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
Code
#28
GAR-BERT
0.49
SUP-EM
No paper
#29
AFSgraph model
0.485
SUP-EM
No paper
#30
GAR
0.483
SUP-EM
No paper
#31
RoBERTa-DenseRetriever-Fast
0.48
SUP-EM
No paper
#32
DPR-recurrent
0.48
SUP-EM
No paper
#33
SAFSR model
SOTA
0.48
SUP-EM
· 2018-09-25
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Code
#34
RoBERTa-DenseRetriever
0.479
SUP-EM
No paper
#35
Quark + SemanticRetrievalMRS IR
0.456
SUP-EM
· 2020-04-14
A Simple Yet Strong Pipeline for HotpotQA
#36
Graph-based Recurrent Retriever
0.441
SUP-EM
No paper
#37
GraphRR-Fast
0.429
SUP-EM
No paper
#38
MIR+EPS+BERT
0.428
SUP-EM
No paper
#39
Transformer-XH
0.417
SUP-EM
No paper
#40
DR model
0.416
SUP-EM
No paper
#41
Transformer-XH-final
0.409
SUP-EM
No paper
Code
#42
SemanticRetrievalMRS
0.387
SUP-EM
· 2019-09-17
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale
Code
#43
DrKIT
0.371
SUP-EM
No paper
#44
GoldEn Retriever
0.307
SUP-EM
· 2019-10-15
Answering Complex Open-domain Questions Through Iterative Query Generation
Code
#45
Entity-centric BERT Pipeline
0.263
SUP-EM
No paper
#46
SAFSr-Bert
0.242
SUP-EM
No paper
#47
Cognitive Graph QA
0.228
SUP-EM
· 2019-05-14
Cognitive Graph for Multi-Hop Reading Comprehension at Scale
Code
#48
PR-Bert
0.219
SUP-EM
No paper
#49
MUPPET
0.167
SUP-EM
· 2019-06-15
Multi-Hop Paragraph Retrieval for Open-Domain Question Answering
Code
#50
IKFGraph
0.16
SUP-EM
No paper
#51
HGN Model-reproduce
0.156
SUP-EM
No paper
#52
AnonymousQ
0.153
SUP-EM
No paper
#53
SAQA
0.147
SUP-EM
No paper
#54
QFE
0.142
SUP-EM
· 2019-05-21
Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction
#55
GRN + BERT
0.132
SUP-EM
No paper
#56
KGNN
0.127
SUP-EM
· 2019-11-06
Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network
#57
GRN
0.122
SUP-EM
No paper
#58
SAFSr_model
0.08
SUP-EM
No paper
#59
GAR-NOSF
0.076
SUP-EM
No paper
#60
SuppBERT
0.056
SUP-EM
No paper
#61
Baseline Model
0.039
SUP-EM
· 2018-09-25
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Code
#62
Entity-centric IR
0.001
SUP-EM
No paper
#63
tes
0
SUP-EM
No paper
#64
PromptRank-fewshot-2-demo
0
SUP-EM
No paper
#65
graph-recurrent-retriever+roberta-base w. S/R-pretraining
0
SUP-EM
No paper
#66
TPReasoner w/o BERT
0
SUP-EM
No paper
#67
MultiQA
0
SUP-EM
No paper
#68
DecompRC
0
SUP-EM
No paper
Code
#69
0
SUP-EM
No paper
#70
Mistral multi hop with very large sources
0
SUP-EM
No paper