Hard Sigmoid

GeneralIntroduced 20004 papers
Source Paper

Description

The Hard Sigmoid is an activation function used for neural networks of the form:

f(x)=max⁡(0,min⁡(1,(x+1)2))f\left(x\right) = \max\left(0, \min\left(1,\frac{\left(x+1\right)}{2}\right)\right)

Image Source: Rinat Maksutov

Papers Using This Method

K-textures, a self-supervised hard clustering deep learning algorithm for satellite image segmentation2022-05-18MaskConvNet: Training Efficient ConvNets from Scratch via Budget-constrained Filter Pruning2020-01-01$L_0$-ARM: Network Sparsification via Stochastic Binary Optimization2019-04-09BinaryConnect: Training Deep Neural Networks with binary weights during propagations2015-11-02