ChatGPT (Baseline)

Reported on 4 benchmarks across 2 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing4 results

Semantic ParsingonBIRD (BIg Bench for LaRge-scale Database Grounded Text-to-SQL Evaluation)
Execution Accuracy % (Dev)· 2023-05-04
37.22
best: 74.32 (DSAIR + GPT-4o)
Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs arXiv:2305.03111
Semantic ParsingonBIRD (BIg Bench for LaRge-scale Database Grounded Text-to-SQL Evaluation)
Execution Accuracy % (Test)· 2023-05-04
39.3
best: 75.63 (XiYan-SQL)
Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs arXiv:2305.03111
Text-To-SQLonBIRD (BIg Bench for LaRge-scale Database Grounded Text-to-SQL Evaluation)
Execution Accuracy % (Dev)· 2023-05-04
37.22
best: 74.32 (DSAIR + GPT-4o)
Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs arXiv:2305.03111
Text-To-SQLonBIRD (BIg Bench for LaRge-scale Database Grounded Text-to-SQL Evaluation)
Execution Accuracy % (Test)· 2023-05-04
39.3
best: 75.63 (XiYan-SQL)
Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs arXiv:2305.03111