Description
Accordion is a gradient communication scheduling algorithm that is generic across models while imposing low computational overheads. Accordion inspects the change in the gradient norms to detect critical regimes and adjusts the communication schedule dynamically. Accordion works for both adjusting the gradient compression rate or the batch size without additional parameter tuning.
Papers Using This Method
Rethinking Layered Graphic Design Generation with a Top-Down Approach2025-07-08Origami Single-end Capacitive Sensing for Continuous Shape Estimation of Morphing Structures2023-07-03Accordion: A Communication-Aware Machine Learning Framework for Next Generation Networks2023-01-12Accordion: Adaptive Gradient Communication via Critical Learning Regime Identification2020-10-29