MARLO + Claude 2.1

Reported on 4 benchmarks across 2 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing4 results

Semantic Parsingonspider
Execution Accuracy (Dev)· 2024-10-17
83.6
best: 87.2 (datagpt-sql-7B + InvalidSQL-Feedback)
Learning Metadata-Agnostic Representations for Text-to-SQL In-Context Example Selection arXiv:2410.14049
Semantic Parsingonspider
Execution Accuracy (Test)· 2024-10-17
84
best: 89.65 (XiYan-SQL)
Learning Metadata-Agnostic Representations for Text-to-SQL In-Context Example Selection arXiv:2410.14049
Text-To-SQLonspider
Execution Accuracy (Dev)· 2024-10-17
83.6
best: 87.2 (datagpt-sql-7B + InvalidSQL-Feedback)
Learning Metadata-Agnostic Representations for Text-to-SQL In-Context Example Selection arXiv:2410.14049
Text-To-SQLonspider
Execution Accuracy (Test)· 2024-10-17
84
best: 89.65 (XiYan-SQL)
Learning Metadata-Agnostic Representations for Text-to-SQL In-Context Example Selection arXiv:2410.14049