Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Methods/NT-Xent

NT-Xent

Normalized Temperature-scaled Cross Entropy Loss

GeneralIntroduced 2000251 papers

Description

NT-Xent, or Normalized Temperature-scaled Cross Entropy Loss, is a loss function. Let $\text{sim}\left(\mathbf{u}, \mathbf{v}\right) = \mathbf{u}^{T}\mathbf{v}/||\mathbf{u}|| ||\mathbf{v}||$ denote the cosine similarity between two vectors $\mathbf{u}$ and $\mathbf{v}$ . Then the loss function for a positive pair of examples $\left(i, j\right)$ is :

$\mathbb{l}\_{i,j} = -\log\frac{\exp\left(\text{sim}\left(\mathbf{z}\_{i}, \mathbf{z}\_{j}\right)/\tau\right)}{\sum^{2N}\_{k=1}\mathcal{1}\_{[k\neq{i}]}\exp\left(\text{sim}\left(\mathbf{z}\_{i}, \mathbf{z}\_{k}\right)/\tau\right)}$

where $\mathcal{1}\_{[k\neq{i}]} \in$ { $0, 1$ } is an indicator function evaluating to $\tau$ iff $\left(i, j\right)$ and $\tau$ denotes a temperature parameter. The final loss is computed across all positive pairs, both $\left(i, j\right)$ and $\left(j, i\right)$ , in a mini-batch.

Source: SimCLR

Papers Using This Method

Low-resource keyword spotting using contrastively trained transformer acoustic word embeddings2025-06-21 Probabilistic Variational Contrastive Learning2025-06-11 scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Data2025-06-10 Circumventing Backdoor Space via Weight Symmetry2025-06-09 SSPS: Self-Supervised Positive Sampling for Robust Self-Supervised Speaker Verification2025-05-20 Representation Learning via Non-Contrastive Mutual Information2025-04-23 Impact of Language Guidance: A Reproducibility Study2025-04-10 Unsupervised Detection of Fraudulent Transactions in E-commerce Using Contrastive Learning2025-03-24 A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics2025-03-21 SinSim: Sinkhorn-Regularized SimCLR2025-02-13 Dataset Ownership Verification in Contrastive Pre-trained Models2025-02-11 Self-Supervised Frameworks for Speaker Verification via Bootstrapped Positive Sampling2025-01-29 Bridging Contrastive Learning and Domain Adaptation: Theoretical Perspective and Practical Application2025-01-28 Learning Compact and Robust Representations for Anomaly Detection2025-01-09 Enhancing Contrastive Learning Inspired by the Philosophy of "The Blind Men and the Elephant"2024-12-21 Self-Supervised Radiograph Anatomical Region Classification -- How Clean Is Your Real-World Data?2024-12-20 Maximising Histopathology Segmentation using Minimal Labels via Self-Supervision2024-12-19 AmCLR: Unified Augmented Learning for Cross-Modal Representations2024-12-10 Mitigating Instance-Dependent Label Noise: Integrating Self-Supervised Pretraining with Pseudo-Label Refinement2024-12-06 Tight PAC-Bayesian Risk Certificates for Contrastive Learning2024-12-04