Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer
Huiyuan Lai, Antonio Toral, Malvina Nissim
Abstract
Scarcity of parallel data causes formality style transfer models to have scarce success in preserving content. We show that fine-tuning pre-trained language (GPT-2) and sequence-to-sequence (BART) models boosts content preservation, and that this is possible even with limited amounts of parallel data. Augmenting these models with rewards that target style and content -- the two core aspects of the task -- we achieve a new state-of-the-art.
Related Papers
Transferring Styles for Reduced Texture Bias and Improved Robustness in Semantic Segmentation Networks2025-07-14AnyI2V: Animating Any Conditional Image with Motion Control2025-07-03Hita: Holistic Tokenizer for Autoregressive Image Generation2025-07-03SA-LUT: Spatial Adaptive 4D Look-Up Table for Photorealistic Style Transfer2025-06-16Fine-Grained control over Music Generation with Activation Steering2025-06-11Training-Free Identity Preservation in Stylized Image Generation Using Diffusion Models2025-06-07Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversion2025-06-04SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation2025-06-03