Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

LARS

GeneralIntroduced 200077 papers

Description

Layer-wise Adaptive Rate Scaling, or LARS, is a large batch optimization technique. There are two notable differences between LARS and other adaptive algorithms such as Adam or RMSProp: first, LARS uses a separate learning rate for each layer and not for each weight. And second, the magnitude of the update is controlled with respect to the weight norm for better control of training speed.

$m\_{t} = \beta\_{1}m\_{t-1} + \left(1-\beta\_{1}\right)\left(g\_{t} + \lambda{x\_{t}}\right)$ $x\_{t+1}^{\left(i\right)} = x\_{t}^{\left(i\right)} - \eta\_{t}\frac{\phi\left(|| x\_{t}^{\left(i\right)} ||\right)}{|| m\_{t}^{\left(i\right)} || }m\_{t}^{\left(i\right)}$

Papers Using This Method

Analyzing Breast Cancer Survival Disparities by Race and Demographic Location: A Survival Analysis Approach2025-06-08 Enhancing Federated Survival Analysis through Peer-Driven Client Reputation in Healthcare2025-05-22 Is Self-Supervised Pre-training on Satellite Imagery Better than ImageNet? A Systematic Study with Sentinel-22025-02-15 Self-Supervised Frameworks for Speaker Verification via Bootstrapped Positive Sampling2025-01-29 Tackling Small Sample Survival Analysis via Transfer Learning: A Study of Colorectal Cancer Prognosis2025-01-21 Prediction of Lung Metastasis from Hepatocellular Carcinoma using the SEER Database2025-01-20 Diagnosis and Severity Assessment of Ulcerative Colitis using Self Supervised Learning2024-12-09 SEER: Self-Aligned Evidence Extraction for Retrieval-Augmented Generation2024-10-15 The Informed Elastic Net for Fast Grouped Variable Selection and FDR Control in Genomics Research2024-10-07 Do Not Design, Learn: A Trainable Scoring Function for Uncertainty Estimation in Generative LLMs2024-06-17 Efficient Preference-based Reinforcement Learning via Aligned Experience Estimation2024-05-29 Self-Supervised Multiple Instance Learning for Acute Myeloid Leukemia Classification2024-03-08 SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learning2024-01-24 Survival Analysis of Young Triple-Negative Breast Cancer Patients2024-01-15 Enhancing Contrastive Learning with Efficient Combinatorial Positive Pairing2024-01-11 Quantum Algorithms for the Pathwise Lasso2023-12-21 Feature Extraction for Generative Medical Imaging Evaluation: New Evidence Against an Evolving Trend2023-11-22 SEER : A Knapsack approach to Exemplar Selection for In-Context HybridQA2023-10-10 Revisiting LARS for Large Batch Training Generalization of Neural Networks2023-09-25 Accelerating Large Batch Training via Gradient Signal to Noise Ratio (GSNR)2023-09-24