TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/PGTask: Introducing the Task of Profile Generation from Di...

PGTask: Introducing the Task of Profile Generation from Dialogues

Rui Ribeiro, Joao P. Carvalho, Luísa Coheur

2023-04-13Profile Generation
PaperPDFCode(official)

Abstract

Recent approaches have attempted to personalize dialogue systems by leveraging profile information into models. However, this knowledge is scarce and difficult to obtain, which makes the extraction/generation of profile information from dialogues a fundamental asset. To surpass this limitation, we introduce the Profile Generation Task (PGTask). We contribute with a new dataset for this problem, comprising profile sentences aligned with related utterances, extracted from a corpus of dialogues. Furthermore, using state-of-the-art methods, we provide a benchmark for profile generation on this novel dataset. Our experiments disclose the challenges of profile generation, and we hope that this introduces a new research direction.

Results

TaskDatasetMetricValueModel
Text GenerationPGDatasetBLEU-161.3gpt2-small
Text GenerationPGDatasetBLEU-232.3gpt2-small
Text GenerationPGDatasetBLEU-320.62gpt2-small
Text GenerationPGDatasetBLEU-49.44gpt2-small
Text GenerationPGDatasetBertScore94.39gpt2-small
Text GenerationPGDatasetROUGE-150.07gpt2-small
Text GenerationPGDatasetROUGE-228.31gpt2-small
Text GenerationPGDatasetROUGE-L50gpt2-small
Text GenerationPGDatasetBLEU-159.31gpt2-medium
Text GenerationPGDatasetBLEU-225.94gpt2-medium
Text GenerationPGDatasetBLEU-315.3gpt2-medium
Text GenerationPGDatasetBLEU-49.17gpt2-medium
Text GenerationPGDatasetBertScore94.76gpt2-medium
Text GenerationPGDatasetROUGE-146.32gpt2-medium
Text GenerationPGDatasetROUGE-224.14gpt2-medium
Text GenerationPGDatasetROUGE-L45.88gpt2-medium

Related Papers

Hierarchical Interaction Summarization and Contrastive Prompting for Explainable Recommendations2025-07-08LettinGo: Explore User Profile Generation for Recommendation System2025-06-23Can LLM-Driven Hard Negative Sampling Empower Collaborative Filtering? Findings and Potentials2025-04-07Automating Personalization: Prompt Optimization for Recommendation Reranking2025-04-04Bursting Filter Bubble: Enhancing Serendipity Recommendations with Aligned Large Language Models2025-02-19User Profile with Large Language Models: Construction, Updating, and Benchmarking2025-02-15Solving the Content Gap in Roblox Game Recommendations: LLM-Based Profile Generation and Reranking2025-02-01A Prompting-Based Representation Learning Method for Recommendation with Large Language Models2024-09-25