TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/Fact-based Text Editing

Fact-based Text Editing

Hayate Iso, chao qiao, Hang Li

2020-07-02ACL 2020 6Fact-based Text Editing
PaperPDFCode(official)

Abstract

We propose a novel text editing task, referred to as \textit{fact-based text editing}, in which the goal is to revise a given document to better describe the facts in a knowledge base (e.g., several triples). The task is important in practice because reflecting the truth is a common requirement in text editing. First, we propose a method for automatically generating a dataset for research on fact-based text editing, where each instance consists of a draft text, a revised text, and several facts represented in triples. We apply the method into two public table-to-text datasets, obtaining two new datasets consisting of 233k and 37k instances, respectively. Next, we propose a new neural network architecture for fact-based text editing, called \textsc{FactEditor}, which edits a draft text by referring to given facts using a buffer, a stream, and a memory. A straightforward approach to address the problem would be to employ an encoder-decoder model. Our experimental results on the two datasets show that \textsc{FactEditor} outperforms the encoder-decoder approach in terms of fidelity and fluency. The results also show that \textsc{FactEditor} conducts inference faster than the encoder-decoder approach.

Results

TaskDatasetMetricValueModel
Text GenerationWebEditADD47.69FactEditor
Text GenerationWebEditBLEU75.68FactEditor
Text GenerationWebEditDELETE0.7707FactEditor
Text GenerationWebEditExact Match24.8FactEditor
Text GenerationWebEditF193.17FactEditor
Text GenerationWebEditKEEP0.9184FactEditor
Text GenerationWebEditPrecision96.88FactEditor
Text GenerationWebEditRecall89.74FactEditor
Text GenerationWebEditSARI72.2FactEditor
Text GenerationWebEditADD43.82EncDecEditor
Text GenerationWebEditBLEU71.03EncDecEditor
Text GenerationWebEditDELETE0.7548EncDecEditor
Text GenerationWebEditExact Match20.96EncDecEditor
Text GenerationWebEditF192.51EncDecEditor
Text GenerationWebEditKEEP0.8949EncDecEditor
Text GenerationWebEditPrecision98.06EncDecEditor
Text GenerationWebEditRecall87.56EncDecEditor
Text GenerationWebEditSARI69.59EncDecEditor
Text GenerationWebEditADD27.86Table-to-Text
Text GenerationWebEditBLEU33.75Table-to-Text
Text GenerationWebEditDELETE0.5219Table-to-Text
Text GenerationWebEditExact Match5.78Table-to-Text
Text GenerationWebEditF190.4Table-to-Text
Text GenerationWebEditKEEP0.5144Table-to-Text
Text GenerationWebEditPrecision98.23Table-to-Text
Text GenerationWebEditRecall83.72Table-to-Text
Text GenerationWebEditSARI43.83Table-to-Text
Text GenerationWebEditADD25.77Text-to-Text
Text GenerationWebEditBLEU63.61Text-to-Text
Text GenerationWebEditDELETE0.678Text-to-Text
Text GenerationWebEditExact Match6.22Text-to-Text
Text GenerationWebEditF179.48Text-to-Text
Text GenerationWebEditKEEP0.8262Text-to-Text
Text GenerationWebEditPrecision81.93Text-to-Text
Text GenerationWebEditRecall77.16Text-to-Text
Text GenerationWebEditSARI58.73Text-to-Text
Text GenerationWebEditADD3.91No-Editing
Text GenerationWebEditBLEU66.67No-Editing
Text GenerationWebEditDELETE0.1202No-Editing
Text GenerationWebEditF180.21No-Editing
Text GenerationWebEditKEEP0.7862No-Editing
Text GenerationWebEditPrecision84.49No-Editing
Text GenerationWebEditRecall76.34No-Editing
Text GenerationWebEditSARI31.51No-Editing
Text GenerationRotoEditADD41.5FactEditor
Text GenerationRotoEditBLEU84.43FactEditor
Text GenerationRotoEditDELETE84.24FactEditor
Text GenerationRotoEditExact Match2.65FactEditor
Text GenerationRotoEditF163.39FactEditor
Text GenerationRotoEditKEEP98.41FactEditor
Text GenerationRotoEditPrecision78.84FactEditor
Text GenerationRotoEditRecall52.3FactEditor
Text GenerationRotoEditSARI74.72FactEditor