Description
A new kind of implicit models, where the output of the network is defined as the solution to an "infinite-level" fixed point equation. Thanks to this we can compute the gradient of the output without activations and therefore with a significantly reduced memory footprint.
Papers Using This Method
Deep Equilibrium models for Poisson Imaging Inverse problems via Mirror Descent2025-07-15Restarted contractive operators to learn at equilibrium2025-06-16DDEQs: Distributional Deep Equilibrium Models through Wasserstein Gradient Flows2025-03-03Neural Cellular Automata and Deep Equilibrium Models2025-01-07A Peaceman-Rachford Splitting Approach with Deep Equilibrium Network for Channel Estimation2024-10-31Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective2024-10-30Infusing Self-Consistency into Density Functional Theory Hamiltonian Prediction via Deep Equilibrium Models2024-06-06Positive concave deep equilibrium models2024-02-06Deep Equilibrium Models are Almost Equivalent to Not-so-deep Explicit Models for High-dimensional Gaussian Mixtures2024-02-05One-Step Diffusion Distillation via Deep Equilibrium Models2023-12-12Accelerating Hopfield Network Dynamics: Beyond Synchronous Updates and Forward Euler2023-11-27Deep Equilibrium Diffusion Restoration with Parallel Sampling2023-11-20TorchDEQ: A Library for Deep Equilibrium Models2023-10-28On the Neural Tangent Kernel of Equilibrium Models2023-10-21Wide Neural Networks as Gaussian Processes: Lessons from Deep Equilibrium Models2023-10-16Deep Equilibrium Object Detection2023-08-18Revisiting Implicit Models: Sparsity Trade-offs Capability in Weight-tied Model for Vision Tasks2023-07-16Deep Equilibrium Multimodal Fusion2023-06-29Improving Adversarial Robustness of DEQs with Explicit Regulations Along the Neural Dynamics2023-06-02A Closer Look at the Adversarial Robustness of Deep Equilibrium Models2023-06-02