Bobo Li, Hao Fei, Fei Li, Yuhan Wu, Jinsong Zhang, Shengqiong Wu, Jingye Li, Yijiang Liu, Lizi Liao, Tat-Seng Chua, Donghong Ji
The rapid development of aspect-based sentiment analysis (ABSA) within recent decades shows great potential for real-world society. The current ABSA works, however, are mostly limited to the scenario of a single text piece, leaving the study in dialogue contexts unexplored. To bridge the gap between fine-grained sentiment analysis and conversational opinion mining, in this work, we introduce a novel task of conversational aspect-based sentiment quadruple analysis, namely DiaASQ, aiming to detect the quadruple of target-aspect-opinion-sentiment in a dialogue. We manually construct a large-scale high-quality DiaASQ dataset in both Chinese and English languages. We deliberately develop a neural model to benchmark the task, which advances in effectively performing end-to-end quadruple prediction, and manages to incorporate rich dialogue-specific and discourse feature representations for better cross-utterance quadruple extraction. We hope the new benchmark will spur more advancements in the sentiment analysis community.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Sentiment Analysis | DiaASQ (ZH) | Pair F1 (aspect-opinion) | 45.44 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (ZH) | Pair F1 (target-aspect) | 48.61 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (ZH) | Pair F1 (target-opinion) | 43.31 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (ZH) | Quad F1 (identification) | 37.51 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (ZH) | Quad F1 (micro) | 34.94 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (ZH) | Span F1 (aspect) | 76.94 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (ZH) | Span F1 (opinion) | 59.35 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (ZH) | Span F1 (target) | 90.23 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (EN) | Pair F1 (aspect-opinion) | 44.27 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (EN) | Pair F1 (target-aspect) | 47.91 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (EN) | Pair F1 (target-opinion) | 45.58 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (EN) | Quad F1 (identification) | 36.8 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (EN) | Quad F1 (micro) | 33.31 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (EN) | Span F1 (aspect) | 74.71 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (EN) | Span F1 (opinion) | 60.22 | E2E-DiaASQ |
| Sentiment Analysis | DiaASQ (EN) | Span F1 (target) | 88.62 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (ZH) | Pair F1 (aspect-opinion) | 45.44 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (ZH) | Pair F1 (target-aspect) | 48.61 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (ZH) | Pair F1 (target-opinion) | 43.31 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (ZH) | Quad F1 (identification) | 37.51 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (ZH) | Quad F1 (micro) | 34.94 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (ZH) | Span F1 (aspect) | 76.94 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (ZH) | Span F1 (opinion) | 59.35 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (ZH) | Span F1 (target) | 90.23 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (EN) | Pair F1 (aspect-opinion) | 44.27 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (EN) | Pair F1 (target-aspect) | 47.91 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (EN) | Pair F1 (target-opinion) | 45.58 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (EN) | Quad F1 (identification) | 36.8 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (EN) | Quad F1 (micro) | 33.31 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (EN) | Span F1 (aspect) | 74.71 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (EN) | Span F1 (opinion) | 60.22 | E2E-DiaASQ |
| Aspect-Based Sentiment Analysis (ABSA) | DiaASQ (EN) | Span F1 (target) | 88.62 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (ZH) | Pair F1 (aspect-opinion) | 45.44 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (ZH) | Pair F1 (target-aspect) | 48.61 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (ZH) | Pair F1 (target-opinion) | 43.31 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (ZH) | Quad F1 (identification) | 37.51 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (ZH) | Quad F1 (micro) | 34.94 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (ZH) | Span F1 (aspect) | 76.94 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (ZH) | Span F1 (opinion) | 59.35 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (ZH) | Span F1 (target) | 90.23 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (EN) | Pair F1 (aspect-opinion) | 44.27 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (EN) | Pair F1 (target-aspect) | 47.91 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (EN) | Pair F1 (target-opinion) | 45.58 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (EN) | Quad F1 (identification) | 36.8 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (EN) | Quad F1 (micro) | 33.31 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (EN) | Span F1 (aspect) | 74.71 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (EN) | Span F1 (opinion) | 60.22 | E2E-DiaASQ |
| Aspect-Category-Opinion-Sentiment Quadruple Extraction | DiaASQ (EN) | Span F1 (target) | 88.62 | E2E-DiaASQ |