Clustering the Sketch: A Novel Approach to Embedding Table Compression

Henry Ling-Hei Tsang, Thomas Dybdahl Ahle

2022-10-12Click-Through Rate Prediction Quantization Clustering Recommendation Systems

Abstract

Embedding tables are used by machine learning systems to work with categorical features. In modern Recommendation Systems, these tables can be very large, necessitating the development of new methods for fitting them in memory, even during training. We suggest Clustered Compositional Embeddings (CCE) which combines clustering-based compression like quantization to codebooks with dynamic methods like The Hashing Trick and Compositional Embeddings (Shi et al., 2020). Experimentally CCE achieves the best of both worlds: The high compression rate of codebook-based quantization, but *dynamically* like hashing-based methods, so it can be used during training. Theoretically, we prove that CCE is guaranteed to converge to the optimal codebook and give a tight bound for the number of iterations required.

Results

Task	Dataset	Metric	Value	Model
Click-Through Rate Prediction	Criteo	AUC	0.806	Clustered Compositional Embeddings
Click-Through Rate Prediction	Criteo	Log Loss	0.449	Clustered Compositional Embeddings

Related Papers

Efficient Deployment of Spiking Neural Networks on SpiNNaker2 for DVS Gesture Recognition Using Neuromorphic Intermediate Representation2025-09-04 An End-to-End DNN Inference Framework for the SpiNNaker2 Neuromorphic MPSoC2025-07-18 Tri-Learn Graph Fusion Network for Attributed Graph Clustering2025-07-18 IP2: Entity-Guided Interest Probing for Personalized News Recommendation2025-07-18 A Reproducibility Study of Product-side Fairness in Bundle Recommendation2025-07-18 Task-Specific Audio Coding for Machines: Machine-Learned Latent Features Are Codes for That Machine2025-07-17 Angle Estimation of a Single Source with Massive Uniform Circular Arrays2025-07-17 SGCL: Unifying Self-Supervised and Supervised Learning for Graph Recommendation2025-07-17