Spider-Agent + Llama-3.1-405B

Reported on 2 benchmarks across 2 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing2 results

Semantic ParsingonSpider 2.0
Success Rate· 2024-11-12
2.21
best: 17.03 (Spider-Agent + o1-preview)
Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows arXiv:2411.07763
Text-To-SQLonSpider 2.0
Success Rate· 2024-11-12
2.21
best: 17.03 (Spider-Agent + o1-preview)
Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows arXiv:2411.07763