Research map / Architectures and efficiency / RNNs and normalization
Accelerating Deep Learning with Millions of Classes
Zhuoning Yuan, Zhishuai Guo, Xiaotian Yu, Xiaoyu Wang, Tianbao Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase random projection size m from 10 to 100 (ablation) | RP softmax · MS-Celeb-1M average verification | accuracy: 0.934 -> 0.965 | Improved |
| increase random projection size m from 100 to 1000 (ablation) | RP softmax · MS-Celeb-1M average verification | accuracy: 0.965 -> 0.960 | No clear effect |
| increase random projection size m from 10 to 500 (ablation) | RP softmax · Penn Treebank | perplexity: 137.59 -> 83.98 | Improved |
| increase random projection size m from 10 to 500 (ablation) | RP softmax · Google Billion Words | perplexity: 122.16 -> 46.34 | Improved |
| add random projected softmax in place of full softmax | ResNet50 · MegaFace2 LFW | verification accuracy: 95.58 -> 96.60 | Improved |
| add random projected softmax in place of full softmax | ResNet50 · ImageNet val | top-1 accuracy: 75.69 -> 73.98 | Hurt |
| add random projected softmax with sampled softmax | LSTM · Google Billion Words | speedup: 2.97 -> 12.5 | Improved |
| add recovery of full softmax classifier after projected training (ablation) | RP softmax · Penn Treebank | perplexity: 83.98 -> 74.78 | Improved |
| add recovery of full softmax classifier after projected training (ablation) | RP softmax · ImageNet val | top-1 accuracy: 73.98 -> 74.56 | Improved |
| add recovery of full softmax classifier after projected training (ablation) | RP softmax · Google Billion Words | perplexity: 46.38 -> 46.56 | No clear effect |
| add adaptive projection size (larger for head classes) to RP (ablation) | RP softmax · Google Billion Words | perplexity: 46.38 -> 74.3 | Hurt |
| add adaptive projection size (larger for head classes) to RP (ablation) | RP softmax · MS-Celeb-1M AgeDB | verification accuracy: 95.22 -> 95.42 | Improved |
Closest papers by meaning
- Accelerated Training for Massive Classification via Dynamic Class Selection (AAAI 2018 · similarity 0.87)
- A Tale of Two Efficient and Informative Negative Sampling Distributions (ICML 2021 · similarity 0.84)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.83)
- DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing (AAAI 2024 · similarity 0.83)
- Accelerating Neural Field Training via Soft Mining (CVPR 2024 · similarity 0.83)
- Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertainty (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
