L1 Regularization

$L_{1}$ Regularization is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the $L\_{1}$ Norm of the weights:

$L\_{new}\left(w\right) = L\_{original}\left(w\right) + \lambda{||w||}\_{1}$

where $\lambda$ is a value determining the strength of the penalty. In contrast to weight decay, $L_{1}$ regularization promotes sparsity; i.e. some parameters have an optimal value of zero.

Image Source: Wikipedia