Semantic Parsing on spider

Metric: Exact Match Accuracy (Dev) (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	Exact Match Accuracy (Dev)▼	Extra Data	Paper	Date↕	Code
1	datagpt-sql-7B + InvalidSQL-Feedback	81.6	Yes	DataGpt-SQL-7B: An Open-Source Language Model fo...	2024-09-24	-
2	RESDSQL-3B + NatSQL	80.5	No	RESDSQL: Decoupling Schema Linking and Skeleton ...	2023-02-12	Code
3	datagpt-sql-7B	80.3	Yes	DataGpt-SQL-7B: An Open-Source Language Model fo...	2024-09-24	-
4	T5-SR	77.2	No	T5-SR: A Unified Seq-to-Seq Decoding Strategy fo...	2023-06-14	Code
5	Graphix-3B+PICARD	77.1	No	Graphix-T5: Mixing Pre-Trained Transformers with...	2023-01-18	Code
6	T5-3B + PICARD	75.5	No	PICARD: Parsing Incrementally for Constrained Au...	2021-09-10	Code
7	DAIL-SQL + GPT-4 + Self-Consistency	74.4	No	Text-to-SQL Empowered by Large Language Models: ...	2023-08-29	Code
8	SADGA + GAP	73.1	No	SADGA: Structure-Aware Dual Graph Aggregation Ne...	2021-11-01	Code
9	RATSQL + GAP	71.8	No	Learning Contextual Representations for Semantic...	2020-12-18	Code
10	T5-3B+NatSQL+Token Preprocessing	69.4	No	Improving Generalization in Language Model-Based...	2023-05-27	Code
11	RYANSQL (BERT)	66.6	No	RYANSQL: Recursively Applying Sketch-based Slot ...	2020-04-07	Code
12	MAPO + TABERTLarge (K = 3)	64.5	No	TaBERT: Pretraining for Joint Understanding of T...	2020-05-17	Code

#1datagpt-sql-7B + InvalidSQL-FeedbackSOTA
81.6
Exact Match Accuracy (Dev)· Extra Data· 2024-09-24
DataGpt-SQL-7B: An Open-Source Language Model for Text-to-SQL
#2RESDSQL-3B + NatSQLSOTA
80.5
Exact Match Accuracy (Dev)· 2023-02-12
RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL Code
#3datagpt-sql-7B
80.3
Exact Match Accuracy (Dev)· Extra Data· 2024-09-24
DataGpt-SQL-7B: An Open-Source Language Model for Text-to-SQL
#4T5-SR
77.2
Exact Match Accuracy (Dev)· 2023-06-14
T5-SR: A Unified Seq-to-Seq Decoding Strategy for Semantic Parsing Code
#5Graphix-3B+PICARDSOTA
77.1
Exact Match Accuracy (Dev)· 2023-01-18
Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing Code
#6T5-3B + PICARDSOTA
75.5
Exact Match Accuracy (Dev)· 2021-09-10
PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models Code
#7DAIL-SQL + GPT-4 + Self-Consistency
74.4
Exact Match Accuracy (Dev)· 2023-08-29
Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation Code
#8SADGA + GAP
73.1
Exact Match Accuracy (Dev)· 2021-11-01
SADGA: Structure-Aware Dual Graph Aggregation Network for Text-to-SQL Code
#9RATSQL + GAPSOTA
71.8
Exact Match Accuracy (Dev)· 2020-12-18
Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training Code
#10T5-3B+NatSQL+Token Preprocessing
69.4
Exact Match Accuracy (Dev)· 2023-05-27
Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques Code
#11RYANSQL (BERT)SOTA
66.6
Exact Match Accuracy (Dev)· 2020-04-07
RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases Code
#12MAPO + TABERTLarge (K = 3)
64.5
Exact Match Accuracy (Dev)· 2020-05-17
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data Code