AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

The trade-offs of model size in large recommendation models : 100GB to 10MB Criteo-tb DLRM model

Aditya Desai, Anshumali Shrivastava

NeurIPS 2022 · Architectures and efficiency › RNNs and normalization · approach: Neural scaling laws · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
compress embeddings 10000x with PSSDLRM · Criteo-TBAUC target 0.8025No clear effect
compress embeddings 100000x with PSS (ablation)DLRM · Criteo-TBtarget AUC reachedHurt
compress embeddings 1000x with PSS (ablation)DLRM · Criteo-KaggleAUC: 0.8031 -> 0.8048No clear effect
compress embeddings 10000x with PSS (ablation)DLRM · Criteo-KaggleAUC: 0.8031 -> 0.8001Hurt
compress embeddings 10000x with PSS (ablation)XDEEPFM · Criteo-KaggleAUC: 0.8007 -> 0.795Hurt
compress embeddings 10000x with PSS (ablation)FIBINET · Criteo-KaggleAUC: 0.8016 -> 0.7963Hurt
increase compression from original to 10000x (ablation)DLRM · Criteo-TBepochs to target AUC: 1.00 -> 4.55Hurt
increase compression from original to 10000x (ablation)DLRM · Criteo-TBtime per 1000 iterations: 50.6 -> 11.6Improved
increase compression from original to 10000x (ablation)DLRM · Criteo-TBrelative total training time: 1 -> 1.04No clear effect
replace multi-GPU original model with PSS on single GPU (ablation)DLRM · Criteo-TB inference 89M samplesinference time: 203 seconds -> 97 secondsImproved
use dense instead of sparse gradients at high compression (ablation)PSS · embedding lookup benchmarkbackward pass latency: up to 8× fasterImproved
hash chunks of size 32 instead of individual elements (ablation)PSS · embedding lookup benchmarklatencyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy