Text-To-SQL on spider

Metric: Execution Accuracy (Test) (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	Execution Accuracy (Test)▼	Extra Data	Paper	Date↕	Code
1	XiYan-SQL	89.65	No	A Preview of XiYan-SQL: A Multi-Generator Ensemb...	2024-11-13	Code
2	PET-SQL	87.6	No	PET-SQL: A Prompt-Enhanced Two-Round Refinement ...	2024-03-13	Code
3	DAIL-SQL + GPT-4 + Self-Consistency	86.6	No	Text-to-SQL Empowered by Large Language Models: ...	2023-08-29	Code
4	DIN-SQL + GPT-4	85.3	No	DIN-SQL: Decomposed In-Context Learning of Text-...	2023-04-21	Code
5	MSc-SQL	84.7	No	MSc-SQL: Multi-Sample Critiquing Small Language ...	2024-10-16	Code
6	MARLO + Claude 2.1	84	No	Learning Metadata-Agnostic Representations for T...	2024-10-17	-
7	C3 + ChatGPT + Zero-Shot	82.3	No	C3: Zero-shot Text-to-SQL with ChatGPT	2023-07-14	Code
8	RESDSQL-3B + NatSQL	79.9	No	RESDSQL: Decoupling Schema Linking and Skeleton ...	2023-02-12	Code
9	T5-3B+NatSQL+Token Preprocessing	78	No	Improving Generalization in Language Model-Based...	2023-05-27	Code
10	Graphix-3B+PICARD	77.6	No	Graphix-T5: Mixing Pre-Trained Transformers with...	2023-01-18	Code
11	T5-SR	75.2	No	T5-SR: A Unified Seq-to-Seq Decoding Strategy fo...	2023-06-14	Code
12	T5-3B + PICARD	75.1	No	PICARD: Parsing Incrementally for Constrained Au...	2021-09-10	Code

#1XiYan-SQLSOTA
89.65
Execution Accuracy (Test)· 2024-11-13
A Preview of XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL Code
#2PET-SQLSOTA
87.6
Execution Accuracy (Test)· 2024-03-13
PET-SQL: A Prompt-Enhanced Two-Round Refinement of Text-to-SQL with Cross-consistency Code
#3DAIL-SQL + GPT-4 + Self-ConsistencySOTA
86.6
Execution Accuracy (Test)· 2023-08-29
Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation Code
#4DIN-SQL + GPT-4SOTA
85.3
Execution Accuracy (Test)· 2023-04-21
DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction Code
#5MSc-SQL
84.7
Execution Accuracy (Test)· 2024-10-16
MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation Code
#6MARLO + Claude 2.1
84
Execution Accuracy (Test)· 2024-10-17
Learning Metadata-Agnostic Representations for Text-to-SQL In-Context Example Selection
#7C3 + ChatGPT + Zero-Shot
82.3
Execution Accuracy (Test)· 2023-07-14
C3: Zero-shot Text-to-SQL with ChatGPT Code
#8RESDSQL-3B + NatSQLSOTA
79.9
Execution Accuracy (Test)· 2023-02-12
RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL Code
#9T5-3B+NatSQL+Token Preprocessing
78
Execution Accuracy (Test)· 2023-05-27
Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques Code
#10Graphix-3B+PICARDSOTA
77.6
Execution Accuracy (Test)· 2023-01-18
Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing Code
#11T5-SR
75.2
Execution Accuracy (Test)· 2023-06-14
T5-SR: A Unified Seq-to-Seq Decoding Strategy for Semantic Parsing Code
#12T5-3B + PICARDSOTA
75.1
Execution Accuracy (Test)· 2021-09-10
PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models Code