CosLU

The Cosine Linear Unit, or CosLU, is a type of activation function that has trainable parameters and uses the cosine function.

$CosLU(x) = (x + \alpha \cos(\beta x))\sigma(x)$