TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Methods/Gumbel Activation

Gumbel Activation

Gumbel Cross Entropy

GeneralIntroduced 20001 papers
Source Paper

Description

Gumbel activation function, is defined using the cumulative Gumbel distribution and it can be used to perform Gumbel regression. Gumbel activation is an alternative activation function to the sigmoid or softmax activation functions and can be used to transform the unormalised output of a model to probability. Gumbel activation ηGumbel\eta_{Gumbel}ηGumbel​ is defined as follows:

ηGumbel(qi)=exp(−exp(−qi))\eta_{Gumbel}(q_i) = exp(-exp(-q_i))ηGumbel​(qi​)=exp(−exp(−qi​))

It can be combined with Cross Entropy loss function to solve long-tailed classification problems. Gumbel Cross Entropy (GCE) is defined as follows:

GCE(ηGumbel(qi),yi)=−yilog⁡(ηGumbel(qi))+(1−yi)log⁡(1−ηGumbel(qi))GCE(\eta_{Gumbel}(q_i),y_i) = -y_i \log(\eta_{Gumbel}(q_i))+ (1-y_i) \log(1-\eta_{Gumbel}(q_i))GCE(ηGumbel​(qi​),yi​)=−yi​log(ηGumbel​(qi​))+(1−yi​)log(1−ηGumbel​(qi​))

Papers Using This Method

Long-tailed Instance Segmentation using Gumbel Optimized Loss2022-07-22