TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/A Slot Is Not Built in One Utterance: Spoken Language Dial...

A Slot Is Not Built in One Utterance: Spoken Language Dialogs with Sub-Slots

Sai Zhang, Yuwei Hu, Yuchuan Wu, Jiaman Wu, Yongbin Li, Jian Sun, Caixia Yuan, Xiaojie Wang

2022-03-21Findings (ACL) 2022 5SSTODTask-Oriented Dialogue Systems
PaperPDFCode(official)

Abstract

A slot value might be provided segment by segment over multiple-turn interactions in a dialog, especially for some important information such as phone numbers and names. It is a common phenomenon in daily life, but little attention has been paid to it in previous work. To fill the gap, this paper defines a new task named Sub-Slot based Task-Oriented Dialog (SSTOD) and builds a Chinese dialog dataset SSD for boosting research on SSTOD. The dataset includes a total of 40K dialogs and 500K utterances from four different domains: Chinese names, phone numbers, ID numbers and license plate numbers. The data is well annotated with sub-slot values, slot values, dialog states and actions. We find some new linguistic phenomena and interactive manners in SSTOD which raise critical challenges of building dialog agents for the task. We test three state-of-the-art dialog models on SSTOD and find they cannot handle the task well on any of the four domains. We also investigate an improved model by involving slot knowledge in a plug-in manner. More work should be done to meet the new challenges raised from SSTOD which widely exists in real-life applications. The dataset and code are publicly available via https://github.com/shunjiu/SSTOD.

Results

TaskDatasetMetricValueModel
DialogueautomataDialogue Success Rate45.8UBAR+
DialogueSSD_NAMEDialogue Success Rate57.73UBAR+
DialogueSSD_NAMEJoint Acc84.96UBAR+
DialogueSSD_NAMESlot Acc93.12UBAR+
Task-Oriented Dialogue SystemsautomataDialogue Success Rate45.8UBAR+
Task-Oriented Dialogue SystemsSSD_NAMEDialogue Success Rate57.73UBAR+
Task-Oriented Dialogue SystemsSSD_NAMEJoint Acc84.96UBAR+
Task-Oriented Dialogue SystemsSSD_NAMESlot Acc93.12UBAR+

Related Papers

An Efficient Task-Oriented Dialogue Policy: Evolutionary Reinforcement Learning Injected by Elite Individuals2025-06-04WHEN TO ACT, WHEN TO WAIT: Modeling Structural Trajectories for Intent Triggerability in Task-Oriented Dialogue2025-06-02EnSToM: Enhancing Dialogue Systems with Entropy-Scaled Steering Vectors for Topic Maintenance2025-05-22clem:todd: A Framework for the Systematic Benchmarking of LLM-Based Task-Oriented Dialogue System Realisations2025-05-08LANID: LLM-assisted New Intent Discovery2025-03-31Interpretable and Robust Dialogue State Tracking via Natural Language Summarization with LLMs2025-03-11Leveraging Graph Structures and Large Language Models for End-to-End Synthetic Task-Oriented Dialogues2025-01-21Intent-driven In-context Learning for Few-shot Dialogue State Tracking2024-12-04