TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/Text is no more Enough! A Benchmark for Profile-based Spok...

Text is no more Enough! A Benchmark for Profile-based Spoken Language Understanding

Xiao Xu, Libo Qin, Kaiji Chen, Guoxing Wu, Linlin Li, Wanxiang Che

2021-12-22Semantic Frame ParsingIntent Detectionslot-fillingSlot FillingSpoken Language Understanding
PaperPDFCode(official)

Abstract

Current researches on spoken language understanding (SLU) heavily are limited to a simple setting: the plain text-based SLU that takes the user utterance as input and generates its corresponding semantic frames (e.g., intent and slots). Unfortunately, such a simple setting may fail to work in complex real-world scenarios when an utterance is semantically ambiguous, which cannot be achieved by the text-based SLU models. In this paper, we first introduce a new and important task, Profile-based Spoken Language Understanding (ProSLU), which requires the model that not only relies on the plain text but also the supporting profile information to predict the correct intents and slots. To this end, we further introduce a large-scale human-annotated Chinese dataset with over 5K utterances and their corresponding supporting profile information (Knowledge Graph (KG), User Profile (UP), Context Awareness (CA)). In addition, we evaluate several state-of-the-art baseline models and explore a multi-level knowledge adapter to effectively incorporate profile information. Experimental results reveal that all existing text-based SLU models fail to work when the utterances are semantically ambiguous and our proposed framework can effectively fuse the supporting information for sentence-level intent detection and token-level slot filling. Finally, we summarize key challenges and provide new points for future directions, which hopes to facilitate the research.

Results

TaskDatasetMetricValueModel
Slot FillingProSLUF10.8327General SLU Model w/ Profile
Intent DetectionProSLUAccuracy0.8531General SLU Model w/ Profile

Related Papers

Flippi: End To End GenAI Assistant for E-Commerce2025-07-08An Interdisciplinary Review of Commonsense Reasoning and Intent Detection2025-06-16Invocable APIs derived from NL2SQL datasets for LLM Tool-Calling Evaluation2025-06-12Integration of Old and New Knowledge for Generalized Intent Discovery: A Consistency-driven Prototype-Prompting Framework2025-06-10MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmark2025-06-05Building a Few-Shot Cross-Domain Multilingual NLU Model for Customer Care2025-06-04ALAS: Measuring Latent Speech-Text Alignment For Spoken Language Understanding In Multimodal LLMs2025-05-26"KAN you hear me?" Exploring Kolmogorov-Arnold Networks for Spoken Language Understanding2025-05-26