Hayate Iso, chao qiao, Hang Li
We propose a novel text editing task, referred to as \textit{fact-based text editing}, in which the goal is to revise a given document to better describe the facts in a knowledge base (e.g., several triples). The task is important in practice because reflecting the truth is a common requirement in text editing. First, we propose a method for automatically generating a dataset for research on fact-based text editing, where each instance consists of a draft text, a revised text, and several facts represented in triples. We apply the method into two public table-to-text datasets, obtaining two new datasets consisting of 233k and 37k instances, respectively. Next, we propose a new neural network architecture for fact-based text editing, called \textsc{FactEditor}, which edits a draft text by referring to given facts using a buffer, a stream, and a memory. A straightforward approach to address the problem would be to employ an encoder-decoder model. Our experimental results on the two datasets show that \textsc{FactEditor} outperforms the encoder-decoder approach in terms of fidelity and fluency. The results also show that \textsc{FactEditor} conducts inference faster than the encoder-decoder approach.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Text Generation | WebEdit | ADD | 47.69 | FactEditor |
| Text Generation | WebEdit | BLEU | 75.68 | FactEditor |
| Text Generation | WebEdit | DELETE | 0.7707 | FactEditor |
| Text Generation | WebEdit | Exact Match | 24.8 | FactEditor |
| Text Generation | WebEdit | F1 | 93.17 | FactEditor |
| Text Generation | WebEdit | KEEP | 0.9184 | FactEditor |
| Text Generation | WebEdit | Precision | 96.88 | FactEditor |
| Text Generation | WebEdit | Recall | 89.74 | FactEditor |
| Text Generation | WebEdit | SARI | 72.2 | FactEditor |
| Text Generation | WebEdit | ADD | 43.82 | EncDecEditor |
| Text Generation | WebEdit | BLEU | 71.03 | EncDecEditor |
| Text Generation | WebEdit | DELETE | 0.7548 | EncDecEditor |
| Text Generation | WebEdit | Exact Match | 20.96 | EncDecEditor |
| Text Generation | WebEdit | F1 | 92.51 | EncDecEditor |
| Text Generation | WebEdit | KEEP | 0.8949 | EncDecEditor |
| Text Generation | WebEdit | Precision | 98.06 | EncDecEditor |
| Text Generation | WebEdit | Recall | 87.56 | EncDecEditor |
| Text Generation | WebEdit | SARI | 69.59 | EncDecEditor |
| Text Generation | WebEdit | ADD | 27.86 | Table-to-Text |
| Text Generation | WebEdit | BLEU | 33.75 | Table-to-Text |
| Text Generation | WebEdit | DELETE | 0.5219 | Table-to-Text |
| Text Generation | WebEdit | Exact Match | 5.78 | Table-to-Text |
| Text Generation | WebEdit | F1 | 90.4 | Table-to-Text |
| Text Generation | WebEdit | KEEP | 0.5144 | Table-to-Text |
| Text Generation | WebEdit | Precision | 98.23 | Table-to-Text |
| Text Generation | WebEdit | Recall | 83.72 | Table-to-Text |
| Text Generation | WebEdit | SARI | 43.83 | Table-to-Text |
| Text Generation | WebEdit | ADD | 25.77 | Text-to-Text |
| Text Generation | WebEdit | BLEU | 63.61 | Text-to-Text |
| Text Generation | WebEdit | DELETE | 0.678 | Text-to-Text |
| Text Generation | WebEdit | Exact Match | 6.22 | Text-to-Text |
| Text Generation | WebEdit | F1 | 79.48 | Text-to-Text |
| Text Generation | WebEdit | KEEP | 0.8262 | Text-to-Text |
| Text Generation | WebEdit | Precision | 81.93 | Text-to-Text |
| Text Generation | WebEdit | Recall | 77.16 | Text-to-Text |
| Text Generation | WebEdit | SARI | 58.73 | Text-to-Text |
| Text Generation | WebEdit | ADD | 3.91 | No-Editing |
| Text Generation | WebEdit | BLEU | 66.67 | No-Editing |
| Text Generation | WebEdit | DELETE | 0.1202 | No-Editing |
| Text Generation | WebEdit | F1 | 80.21 | No-Editing |
| Text Generation | WebEdit | KEEP | 0.7862 | No-Editing |
| Text Generation | WebEdit | Precision | 84.49 | No-Editing |
| Text Generation | WebEdit | Recall | 76.34 | No-Editing |
| Text Generation | WebEdit | SARI | 31.51 | No-Editing |
| Text Generation | RotoEdit | ADD | 41.5 | FactEditor |
| Text Generation | RotoEdit | BLEU | 84.43 | FactEditor |
| Text Generation | RotoEdit | DELETE | 84.24 | FactEditor |
| Text Generation | RotoEdit | Exact Match | 2.65 | FactEditor |
| Text Generation | RotoEdit | F1 | 63.39 | FactEditor |
| Text Generation | RotoEdit | KEEP | 98.41 | FactEditor |
| Text Generation | RotoEdit | Precision | 78.84 | FactEditor |
| Text Generation | RotoEdit | Recall | 52.3 | FactEditor |
| Text Generation | RotoEdit | SARI | 74.72 | FactEditor |