code-davinci-002 175B (LEVER)

Reported on 3 benchmarks across 2 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing3 results

Semantic Parsingonspider
Accuracy· 2023-02-16
81.9
best: 84.1 (RESDSQL-3B + NatSQL)
LEVER: Learning to Verify Language-to-Code Generation with Execution arXiv:2302.08468
Semantic Parsingonspider
Execution Accuracy (Dev)· 2023-02-16
81.9
best: 87.2 (datagpt-sql-7B + InvalidSQL-Feedback)
LEVER: Learning to Verify Language-to-Code Generation with Execution arXiv:2302.08468
Text-To-SQLonspider
Execution Accuracy (Dev)· 2023-02-16
81.9
best: 87.2 (datagpt-sql-7B + InvalidSQL-Feedback)
LEVER: Learning to Verify Language-to-Code Generation with Execution arXiv:2302.08468