AdaFisher

Adaptive Second Order Optimization via Fisher Information

GeneralIntroduced 20002 papers
Source Paper

Description

AdaFisher – an adaptive second-order optimizer that leverages a block-diagonal approximation to the Fisher information matrix for adaptive gradient preconditioning.

Papers Using This Method

Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information Analysis2025-04-26AdaFisher: Adaptive Second Order Optimization via Fisher Information2024-05-26