Semantic Parsing on LDC2017T10

Metric: Smatch (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	Smatch▼	Extra Data	Paper	Date↕	Code
1	StructBART + MBSE (IBM)	86.7	Yes	Maximum Bayes Smatch Ensemble Distillation for A...	2021-12-14	Code
2	Graphene Smatch (IBM)	86.26	Yes	Ensembling Graph Predictions for AMR Parsing	2021-10-18	Code
3	LeakDistill	86.1	Yes	Incorporating Graph Information in Transformer-b...	2023-06-23	Code
4	StructBART + MBSE (IBM)	85.9	Yes	Maximum Bayes Smatch Ensemble Distillation for A...	2021-12-14	Code
5	Graphene Support (IBM)	85.85	Yes	Ensembling Graph Predictions for AMR Parsing	2021-10-18	Code
6	AMRBART large	85.4	Yes	Graph Pre-training for AMR Parsing and Generation	2022-03-15	Code
7	ATP-SRL (Ensemble)	85.3	Yes	ATP: AMRize Then Parse! Enhancing AMR Parsing wi...	2022-04-19	Code
8	ATP-SRL	85.2	Yes	ATP: AMRize Then Parse! Enhancing AMR Parsing wi...	2022-04-19	Code
9	StructBART-J (IBM)	84.7	Yes	Structure-aware Fine-tuning of Sequence-to-seque...	2021-10-29	Code
10	LeakDistill (base)	84.7	Yes	Incorporating Graph Information in Transformer-b...	2023-06-23	Code
11	BiBL+Silver	84.7	Yes	-	-	Code
12	BiBL	84.6	No	-	-	Code
13	SPRING	84.3	Yes	-	-	Code
14	APT base (IBM)	82.6	Yes	AMR Parsing with Action-Pointer Transformer	2021-04-29	Code
15	Xu, et al.	81.4	Yes	Improving AMR Parsing with Sequence-to-Sequence ...	2020-10-05	Code
16	stack-Transformer + self-learning (IBM)	81.3	Yes	Pushing the Limits of AMR Parsing with Self-Lear...	2020-10-20	Code
17	Cai and Lam	80.2	No	AMR Parsing via Graph-Sequence Iterative Inference	2020-04-12	Code
18	ND+AD+LV	80	No	Levi Graph AMR Parser using Heterogeneous Attent...	2021-07-09	Code
19	stack-Transformer (IBM)	79	No	Transition-based Parsing with Stack-Transformers	2020-10-20	Code
20	Zhang et al.	77	No	Broad-Coverage Semantic Parsing as Transduction	2019-09-05	-
21	Sequence-to-Graph Transduction	76.3	No	AMR Parsing as Sequence-to-Graph Transduction	2019-05-21	Code
22	Lyu et al. 2021. Full	76.1	No	A Differentiable Relaxation of Graph Segmentatio...	2020-10-23	-
23	Joint model	74.4	No	AMR Parsing as Graph Prediction with Latent Alig...	2018-05-14	Code
24	Rewarding Smatch (IBM)	73.4	No	Rewarding Smatch: Transition-Based AMR Parsing w...	2019-05-31	-
25	Cai and Lam	73.2	No	Core Semantic First: A Top-down Approach for AMR...	2019-09-10	Code
26	ChSeq + 100K	71	No	Neural Semantic Parsing by Character-based Trans...	2017-05-28	Code
27	Neural-Pointer	61.9	No	-	-	-

#1StructBART + MBSE (IBM)SOTA
86.7
Smatch· Extra Data· 2021-12-14
Maximum Bayes Smatch Ensemble Distillation for AMR Parsing Code
#2Graphene Smatch (IBM)SOTA
86.26
Smatch· Extra Data· 2021-10-18
Ensembling Graph Predictions for AMR Parsing Code
#3LeakDistill
86.1
Smatch· Extra Data· 2023-06-23
Incorporating Graph Information in Transformer-based AMR Parsing Code
#4StructBART + MBSE (IBM)
85.9
Smatch· Extra Data· 2021-12-14
Maximum Bayes Smatch Ensemble Distillation for AMR Parsing Code
#5Graphene Support (IBM)
85.85
Smatch· Extra Data· 2021-10-18
Ensembling Graph Predictions for AMR Parsing Code
#6AMRBART large
85.4
Smatch· Extra Data· 2022-03-15
Graph Pre-training for AMR Parsing and Generation Code
#7ATP-SRL (Ensemble)
85.3
Smatch· Extra Data· 2022-04-19
ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs Code
#8ATP-SRL
85.2
Smatch· Extra Data· 2022-04-19
ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs Code
#9StructBART-J (IBM)
84.7
Smatch· Extra Data· 2021-10-29
Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing Code
#10LeakDistill (base)
84.7
Smatch· Extra Data· 2023-06-23
Incorporating Graph Information in Transformer-based AMR Parsing Code
#11BiBL+Silver
84.7
Smatch· Extra Data
No paperCode
#12BiBL
84.6
Smatch
No paperCode
#13SPRING
84.3
Smatch· Extra Data
No paperCode
#14APT base (IBM)SOTA
82.6
Smatch· Extra Data· 2021-04-29
AMR Parsing with Action-Pointer Transformer Code
#15Xu, et al.SOTA
81.4
Smatch· Extra Data· 2020-10-05
Improving AMR Parsing with Sequence-to-Sequence Pre-training Code
#16stack-Transformer + self-learning (IBM)
81.3
Smatch· Extra Data· 2020-10-20
Pushing the Limits of AMR Parsing with Self-Learning Code
#17Cai and LamSOTA
80.2
Smatch· 2020-04-12
AMR Parsing via Graph-Sequence Iterative Inference Code
#18ND+AD+LV
80
Smatch· 2021-07-09
Levi Graph AMR Parser using Heterogeneous Attention Code
#19stack-Transformer (IBM)
79
Smatch· 2020-10-20
Transition-based Parsing with Stack-Transformers Code
#20Zhang et al.SOTA
77
Smatch· 2019-09-05
Broad-Coverage Semantic Parsing as Transduction
#21Sequence-to-Graph TransductionSOTA
76.3
Smatch· 2019-05-21
AMR Parsing as Sequence-to-Graph Transduction Code
#22Lyu et al. 2021. Full
76.1
Smatch· 2020-10-23
A Differentiable Relaxation of Graph Segmentation and Alignment for AMR Parsing
#23Joint modelSOTA
74.4
Smatch· 2018-05-14
AMR Parsing as Graph Prediction with Latent Alignment Code
#24Rewarding Smatch (IBM)
73.4
Smatch· 2019-05-31
Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning
#25Cai and Lam
73.2
Smatch· 2019-09-10
Core Semantic First: A Top-down Approach for AMR Parsing Code
#26ChSeq + 100KSOTA
71
Smatch· 2017-05-28
Neural Semantic Parsing by Character-based Translation: Experiments with Abstract Meaning Representations Code
#27Neural-Pointer
61.9
Smatch
No paper