Lecun's Tanh

GeneralIntroduced 1998

Description

LeCun's tanh is an activation function of the form f(x)=1.7159tanh(23x)f\left(x\right) = 1.7159\tanh\left(\frac{2}{3}x\right). The constants were chosen to keep the variance of the output close to 1.