Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

SotA/Natural Language Processing/Semantic Parsing

Semantic Parsing

70 benchmarks1202 papers

Semantic Parsing is the task of transducing natural language utterances into formal meaning representations. The target meaning representations can be defined according to a wide variety of formalisms. This include linguistically-motivated semantic representations that are designed to capture the meaning of any sentence such as λ-calculus or the abstract meaning representations. Alternatively, for more task-driven approaches to Semantic Parsing, it is common for meaning representations to represent executable programs such as SQL queries, robotic commands, smart phone instructions, and even general-purpose programming languages like Python and Java.

<span class="description-source">Source: Tranx: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation </span>

Benchmarks

Semantic Parsing on BIRD (BIg Bench for LaRge-scale Database Grounded Text-to-SQL Evaluation)

Execution Accuracy % (Dev)Execution Accuracy % (Test)Execution Accurarcy (Human)

Semantic Parsing on LDC2017T10

Semantic Parsing on WikiTableQuestions

Accuracy (Test)Accuracy (Dev)Accuracy Test Accuracy

Semantic Parsing on LDC2020T02

Semantic Parsing on spider

Execution Accuracy (Test)Exact Match Accuracy (Dev)Accuracy Execution Accuracy (Dev)Exact Match Accuracy (Test)

Semantic Parsing on LDC2014T12

F1 Full F1 Newswire

Semantic Parsing on Spider 2.0

Semantic Parsing on SParC

interaction match accuracy question match accuracy Exact

Semantic Parsing on LDC2015E86

Semantic Parsing on MMSQL

Semantic Parsing on PMB-2.2.0

Semantic Parsing on Bio

Semantic Parsing on CFQ

Semantic Parsing on LDC2014T12:

F1 Newswire F1 Full

Semantic Parsing on WebQuestionsSP

Semantic Parsing on ATIS

Semantic Parsing on DM

In-domain Out-of-domain

Semantic Parsing on New3

Semantic Parsing on PAS

In-domain Out-of-domain

Semantic Parsing on PSD

In-domain Out-of-domain

Semantic Parsing on SPIDER

Exact Match Accuracy (in Dev)Execution Accuracy (in Dev)

Semantic Parsing on SemEval 2019 Task 1

English-Wiki (open) F1 English-20K (open) F1

Semantic Parsing on The Little Prince

Semantic Parsing on WikiSQL

Denotation accuracy (test)Accuracy

Semantic Parsing on CoNLL 2019

Full UCCA F1 Full MRP F1 LPP UCCA F1 LPP MRP F1

Semantic Parsing on Geo

Semantic Parsing on PMB-3.0.0

Semantic Parsing on PTG (czech, MRP 2020)

Semantic Parsing on AMR (chinese, MRP 2020)

Semantic Parsing on AMR (english, MRP 2020)

Semantic Parsing on DRG (english, MRP 2020)

Semantic Parsing on DRG (german, MRP 2020)

Semantic Parsing on EDS (english, MRP 2020)

Semantic Parsing on KaggleDBQA

Exact Match (EM)

Semantic Parsing on PTG (english, MRP 2020)

Semantic Parsing on UCCA (english, MRP 2020)

Semantic Parsing on UCCA (german, MRP 2020)

Semantic Parsing on GraphQuestions

Semantic Parsing on SEDE

PCM-F1 (dev)PCM-F1 (test)

Semantic Parsing on SQA

Denotation Accuracy Accuracy

Semantic Parsing on SQL-Eval

Execution Accuracy

Semantic Parsing on Text-To-SQL

Semantic Parsing on VG graph-text

Semantic Parsing on WebNLG v2.1

Semantic Parsing on complexWebQuestions-V1.0

Semantic Parsing on 2D KITTI Cars Easy