Liying Cheng, Lidong Bing, Ruidan He, Qian Yu, Yan Zhang, Luo Si
Traditionally, a debate usually requires a manual preparation process, including reading plenty of articles, selecting the claims, identifying the stances of the claims, seeking the evidence for the claims, etc. As the AI debate attracts more attention these years, it is worth exploring the methods to automate the tedious process involved in the debating system. In this work, we introduce a comprehensive and large dataset named IAM, which can be applied to a series of argument mining tasks, including claim extraction, stance classification, evidence extraction, etc. Our dataset is collected from over 1k articles related to 123 topics. Near 70k sentences in the dataset are fully annotated based on their argument properties (e.g., claims, stances, evidence, etc.). We further propose two new integrated argument mining tasks associated with the debate preparation process: (1) claim extraction with stance classification (CESC) and (2) claim-evidence pair extraction (CEPE). We adopt a pipeline approach and an end-to-end method for each integrated task separately. Promising experimental results are reported to show the values and challenges of our proposed tasks, and motivate future research on argument mining.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Data Mining | IAM Dataset | Macro F1 | 60.25 | Multi-label |
| Data Mining | IAM Dataset | F1 | 35.92 | Multi-task |
| Interpretable Machine Learning | IAM Dataset | Macro F1 | 60.25 | Multi-label |
| Interpretable Machine Learning | IAM Dataset | F1 | 35.92 | Multi-task |
| Argument Mining | IAM Dataset | Macro F1 | 60.25 | Multi-label |
| Argument Mining | IAM Dataset | F1 | 35.92 | Multi-task |