AI Paper Atlas

Research map / Learning theory and algorithms / Clustering

Clustering the Sketch: Dynamic Compression for Embedding Tables

Henry Tsang, Thomas Dybdahl Ahle

NeurIPS 2023 · Learning theory and algorithms › Clustering · approach: Clustering approximation algorithms · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use CCE instead of the hashing trickCCE · Criteo Kaggle up to 10 epochscompression at baseline BCE: 4,600× -> 8,500×Improved
use CCE instead of CE with concatenationCCE · Criteo Kaggle up to 10 epochscompression at baseline BCE: 3,800× -> 8,500×Improved
use CCE instead of deep hash embeddingsCCE · Criteo Kaggle up to 10 epochscompression at baseline BCE: 1,300× -> 8,500×Improved
use CCE instead of the hashing trickCCE · Criteo Kaggle 1 epochcompression at baseline BCE: 78 −122× -> 212×Improved
use CCE instead of the hashing trickCCE · Criteo Terabyte 1 epochcompression at baseline BCE: 23 −32× -> 101×Improved
use CCE instead of CE with concatenationCCE · Criteo Terabyte 1 epochcompression at baseline BCE: 25 −48× -> 101×Improved
train CCE with optimal parameters to convergence vs full table baselineCCE · Criteo Kaggletest BCE: 0.66% lower BCEImproved
replace full embedding table by compressed embeddings when training multiple epochsDLRM · Criteo Kaggletest BCEImproved
fine-tune tables after product quantization (ablation)Product Quantization · Criteo Kaggletest BCEHurt
increase number of clusterings (ct1 to ct6, once per epoch) (ablation)CCE · Criteo Kaggle 10 epochstest BCEImproved
cluster once per epoch for first 6 epochs (ct6 cf300000) (ablation)CCE · Criteo Kaggle 10 epochstest BCEImproved
finish all clusterings by 2/3 of an epoch (Strategy 2) (ablation)CCE · Criteo Kaggle 1 epochtest BCEHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy