Semantic Parsing on spider

Metric: Accuracy (higher is better)

LeaderboardDataset

Loading chart...

Results

Submit a result

Sort:

#	Model↕	Accuracy▼	Extra Data	Paper	Date↕	Code
1	RESDSQL-3B + NatSQL	84.1	No	RESDSQL: Decoupling Schema Linking and Skeleton ...	2023-02-12	Code
2	code-davinci-002 175B (LEVER)	81.9	No	LEVER: Learning to Verify Language-to-Code Gener...	2023-02-16	Code
3	RASAT+PICARD	75.5	No	RASAT: Integrating Relational Structures into Pr...	2022-05-14	Code
4	Graphix-3B + PICARD	74	No	Graphix-T5: Mixing Pre-Trained Transformers with...	2023-01-18	Code
5	T5-3B + PICARD	71.9	No	PICARD: Parsing Incrementally for Constrained Au...	2021-09-10	Code
6	SADGA + GAP	70.1	No	SADGA: Structure-Aware Dual Graph Aggregation Ne...	2021-11-01	Code
7	RATSQL + GAP	69.7	No	Learning Contextual Representations for Semantic...	2020-12-18	Code
8	RATSQL + Grammar-Augmented Pre-Training	69.6	No	GraPPa: Grammar-Augmented Pre-Training for Table...	2020-09-29	Code
9	RATSQL + BERT	65.6	No	RAT-SQL: Relation-Aware Schema Encoding and Link...	2019-11-10	Code
10	Exact Set Matching	19.7	No	Spider: A Large-Scale Human-Labeled Dataset for ...	2018-09-24	Code

#1RESDSQL-3B + NatSQLSOTA
84.1
Accuracy· 2023-02-12
RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL Code
#2code-davinci-002 175B (LEVER)
81.9
Accuracy· 2023-02-16
LEVER: Learning to Verify Language-to-Code Generation with Execution Code
#3RASAT+PICARDSOTA
75.5
Accuracy· 2022-05-14
RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL Code
#4Graphix-3B + PICARD
74
Accuracy· 2023-01-18
Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing Code
#5T5-3B + PICARDSOTA
71.9
Accuracy· 2021-09-10
PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models Code
#6SADGA + GAP
70.1
Accuracy· 2021-11-01
SADGA: Structure-Aware Dual Graph Aggregation Network for Text-to-SQL Code
#7RATSQL + GAPSOTA
69.7
Accuracy· 2020-12-18
Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training Code
#8RATSQL + Grammar-Augmented Pre-TrainingSOTA
69.6
Accuracy· 2020-09-29
GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing Code
#9RATSQL + BERTSOTA
65.6
Accuracy· 2019-11-10
RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers Code
#10Exact Set MatchingSOTA
19.7
Accuracy· 2018-09-24
Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task Code