StarReLU

GeneralIntroduced 20001 papers

Description

s(ReLU(x))2+bs \cdot (\mathrm{ReLU}(x))^2 + b

where sRs \in \mathbb{R} and bRb \in \mathbb{R} are shared for all channels and can be set as constants (s=0.8944, b=-0.4472) or learnable parameters.

Papers Using This Method