Semantic Parsing on spider

Metric: Execution Accuracy (Dev) (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	Execution Accuracy (Dev)▼	Extra Data	Paper	Date↕	Code
1	datagpt-sql-7B + InvalidSQL-Feedback	87.2	Yes	DataGpt-SQL-7B: An Open-Source Language Model fo...	2024-09-24	-
2	datagpt-sql-7B	84.8	Yes	DataGpt-SQL-7B: An Open-Source Language Model fo...	2024-09-24	-
3	DAIL-SQL + GPT-4 + Self-Consistency	84.4	No	Text-to-SQL Empowered by Large Language Models: ...	2023-08-29	Code
4	RESDSQL-3B + NatSQL	84.1	No	RESDSQL: Decoupling Schema Linking and Skeleton ...	2023-02-12	Code
5	MARLO + Claude 2.1	83.6	No	Learning Metadata-Agnostic Representations for T...	2024-10-17	-
6	code-davinci-002 175B (LEVER)	81.9	No	LEVER: Learning to Verify Language-to-Code Gener...	2023-02-16	Code
7	C3 + ChatGPT + Zero-Shot	81.8	No	C3: Zero-shot Text-to-SQL with ChatGPT	2023-07-14	Code
8	Graphix-3B+PICARD	81	No	Graphix-T5: Mixing Pre-Trained Transformers with...	2023-01-18	Code
9	T5-3B+NatSQL+Token Preprocessing	73.7	No	Improving Generalization in Language Model-Based...	2023-05-27	Code
10	DELLM + GPT-4	71.68	No	Knowledge-to-SQL: Enhancing SQL Generation with ...	2024-02-18	Code

#1datagpt-sql-7B + InvalidSQL-FeedbackSOTA
87.2
Execution Accuracy (Dev)· Extra Data· 2024-09-24
DataGpt-SQL-7B: An Open-Source Language Model for Text-to-SQL
#2datagpt-sql-7B
84.8
Execution Accuracy (Dev)· Extra Data· 2024-09-24
DataGpt-SQL-7B: An Open-Source Language Model for Text-to-SQL
#3DAIL-SQL + GPT-4 + Self-ConsistencySOTA
84.4
Execution Accuracy (Dev)· 2023-08-29
Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation Code
#4RESDSQL-3B + NatSQLSOTA
84.1
Execution Accuracy (Dev)· 2023-02-12
RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL Code
#5MARLO + Claude 2.1
83.6
Execution Accuracy (Dev)· 2024-10-17
Learning Metadata-Agnostic Representations for Text-to-SQL In-Context Example Selection
#6code-davinci-002 175B (LEVER)
81.9
Execution Accuracy (Dev)· 2023-02-16
LEVER: Learning to Verify Language-to-Code Generation with Execution Code
#7C3 + ChatGPT + Zero-Shot
81.8
Execution Accuracy (Dev)· 2023-07-14
C3: Zero-shot Text-to-SQL with ChatGPT Code
#8Graphix-3B+PICARDSOTA
81
Execution Accuracy (Dev)· 2023-01-18
Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing Code
#9T5-3B+NatSQL+Token Preprocessing
73.7
Execution Accuracy (Dev)· 2023-05-27
Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques Code
#10DELLM + GPT-4
71.68
Execution Accuracy (Dev)· 2024-02-18
Knowledge-to-SQL: Enhancing SQL Generation with Data Expert LLM Code