Description
Just as dropout prevents co-adaptation of activations, DropPath prevents co-adaptation of parallel paths in networks such as FractalNets by randomly dropping operands of the join layers. This discourages the network from using one input path as an anchor and another as a corrective term (a configuration that, if not prevented, is prone to overfitting). Two sampling strategies are:
- Local: a join drops each input with fixed probability, but we make sure at least one survives.
- Global: a single path is selected for the entire network. We restrict this path to be a single column, thereby promoting individual columns as independently strong predictors.
Papers Using This Method
Masked Transformer for Electrocardiogram Classification2023-08-31Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edge2023-01-24MoBYv2AL: Self-supervised Active Learning for Image Classification2023-01-04NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension2022-11-23Delving into Masked Autoencoders for Multi-Label Thorax Disease Classification2022-10-23Search Space Adaptation for Differentiable Neural Architecture Search in Image Classification2022-06-05AGNAS: Attention-Guided Micro- and Macro-Architecture Search2022-06-05Generalizing Few-Shot NAS with Gradient Matching2022-03-29Towards Tailored Models on Private AIoT Devices: Federated Direct Neural Architecture Search2022-02-23Data-Free Neural Architecture Search via Recursive Label Calibration2021-12-03Profiling Neural Blocks and Design Spaces for Mobile Neural Architecture Search2021-09-25Federated Model Search via Reinforcement Learning2021-07-07Generative Adversarial Neural Architecture Search2021-05-19Self-Supervised Learning with Swin Transformers2021-05-10Generalized Latency Performance Estimation for Once-For-All Neural Architecture Search2021-01-04Efficient Neural Architecture Search for End-to-end Speech Recognition via Straight-Through Gradients2020-11-11How Does Supernet Help in Neural Architecture Search?2020-10-16Angle-based Search Space Shrinking for Neural Architecture Search2020-04-28UNAS: Differentiable Architecture Search Meets Reinforcement Learning2019-12-16Neural Predictor for Neural Architecture Search2019-12-02