Description
Compress an ensemble of models into a single one by averaging their weights (under certain pre-conditions).
Papers Using This Method
UC-MOA: Utility-Conditioned Multi-Objective Alignment for Distributional Pareto-Optimality2025-03-10Local Superior Soups: A Catalyst for Model Merging in Cross-Silo Federated Learning2024-10-31Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation2024-10-10Robust Biharmonic Skinning Using Geometric Fields2024-06-01Cross-Dataset Generalization For Retinal Lesions Segmentation2024-05-14How Much You Ate? Food Portion Estimation on Spoons2024-05-12FissionFusion: Fast Geometric Generation and Hierarchical Souping for Medical Image Analysis2024-03-20RADIN: Souping on a Budget2024-01-31Partial Fine-Tuning: A Successor to Full Fine-Tuning for Vision Transformers2023-12-25Descriptor and Word Soups: Overcoming the Parameter Efficiency Accuracy Tradeoff for Out-of-Distribution Few-shot Learning2023-11-21Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging2023-06-29Graph Ladling: Shockingly Simple Parallel GNN Training without Intermediate Communication2023-06-18Pre-training Language Model as a Multi-perspective Course Learner2023-05-06Seasoning Model Soups for Robustness to Adversarial and Natural Distribution Shifts2023-02-20Candidate Soups: Fusing Candidate Results Improves Translation Quality for Non-Autoregressive Translation2023-01-27Model soups to increase inference without increasing compute time2023-01-24Revisiting adapters with adversarial training2022-10-10Bag of Tricks for Domain Adaptive Multi-Object Tracking2022-05-31Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time2022-03-10