Sophia

Second-order Clipped Stochastic Optimization

GeneralIntroduced 20002 papers
Source Paper

Description

Please enter a description about the method here

Papers Using This Method

CoLLiE: Collaborative Training of Large Language Models in an Efficient Way2023-12-01Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training2023-05-23