Research map / Label-efficient and robust learning / Recognition and long-tail
A Tale of Two Efficient and Informative Negative Sampling Distributions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive LSH Embedding negative sampling instead of full softmax | Large-output neural classifier · Amazon-670K | training time per epoch: 11x | Helped |
| Use adaptive LSH Embedding negative sampling instead of full softmax | Large-output neural classifier · Wiki-325K | training time per epoch: 6.5x | Helped |
| Use adaptive LSH Embedding negative sampling instead of full softmax | Large-output neural classifier · ODP-105K | training time per epoch: 15x | Helped |
| Use adaptive LSH Embedding negative sampling instead of full softmax | Large-output neural classifier · Amazon-Uniform | training time per epoch: 22x | Helped |
| Use LSH Embedding rather than static Sampled Softmax negatives | Large-output classifier · Amazon-Uniform | P@1: 20.7 -> 22.8 | Helped |
| Use LSH Embedding rather than static NCE negatives | Large-output classifier · Amazon-Uniform | P@1: 16.1 -> 22.8 | Helped |
| Reduce LSH Embedding negative-sample budget from 0.05N to 0.005N (ablation) | LSH Embedding · Amazon-670K | P@1: 36.1 -> 33.4 | Hurt |
| Reduce LSH Label negative-sample budget from 0.05N to 0.005N (ablation) | LSH Label · Amazon-670K | P@1: 35.5 -> 32 | Hurt |
| Increase hash-table count from 100 to 200 (ablation) | LSH Embedding · Amazon-670K | P@1: 34.5 -> 35.4 | Helped |
| Increase hash-table count from 200 to 300 (ablation) | LSH Embedding · Amazon-670K | P@1: 35.4 -> 35.7 | Helped |
| Increase hash-table count from 100 to 200 (ablation) | LSH Embedding · Wiki-325K | P@1: 53.4 -> 55.4 | Helped |
| Use 5 hash functions instead of 4 (ablation) | LSH Embedding · Wiki-325K | P@1: 38.4 -> 56.3 | Helped |
Closest papers by meaning
- Extreme Classification via Adversarial Softmax Approximation (ICLR 2020 · similarity 0.90)
- Sampled Softmax with Random Fourier Features (NeurIPS 2019 · similarity 0.88)
- Adaptive Sampling for Efficient Softmax Approximation (NeurIPS 2024 · similarity 0.86)
- Adaptive Hardness Negative Sampling for Collaborative Filtering (AAAI 2024 · similarity 0.84)
- EMC$^2$: Efficient MCMC Negative Sampling for Contrastive Learning with Global Convergence (ICML 2024 · similarity 0.83)
- Density-Softmax: Efficient Test-time Model for Uncertainty Estimation and Robustness under Distribution Shifts (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.