Description
Virtual Data Augmentation, or VDA, is a framework for robustly fine-tuning pre-trained language model. Based on the original token embeddings, a multinomial mixture for augmenting virtual data is constructed, where a masked language model guarantees the semantic relevance and the Gaussian noise provides the augmentation diversity. Furthermore, a regularized training strategy is proposed to balance the two aspects.
Papers Using This Method
Approximate Nullspace Augmented Finetuning for Robust Vision Transformers2024-03-15Intra- & Extra-Source Exemplar-Based Style Synthesis for Improved Domain Generalization2023-07-02Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models2021-09-13Ray: A Distributed Framework for Emerging AI Applications2017-12-16