Research map / Label-efficient and robust learning / Knowledge distillation
Toward Understanding Privileged Features Distillation in Learning-to-Rank
Shuo Yang, sujay sanghavi, Holakou Rahmanian, Jan Bakus, Vishwanathan S. V. N.
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a teacher with regular and privileged features for distillation instead of no distillation | Ranking model · Yahoo, Istella, Web30k | NDCG@8: 0.517 $\pm$ 0.005 (+0.0\%) -> 0.566 $\pm$ 0.004 (+9.5\%); 0.402 $\pm$ 0.001 (+0.0\%) -> 0.417 $\pm$ 0.002 (+3.7\%); 0.24 | Improved |
| Use a teacher with both regular and privileged features rather than a privileged-features-only teacher | Ranking model · Yahoo, Istella, Web30k | NDCG@8: 0.557 -> 0.566; 0.380 -> 0.417; 0.205 -> 0.252 | Mixed |
| Add click as a privileged feature to PFD | Amazon purchase ranking · Amazon search logs | Purchase NDCG@8: 1.000 -> 1.082 | Improved |
| Distill from position, click, and add-to-cart teachers rather than a single teacher | PFD student · Amazon search logs | Purchase NDCG@8: +11.2\% over baseline | Improved |
| Use positive query groups rather than all query groups for the data loss (ablation) | PFD · Yahoo, Istella, Web30k | NDCG@8: Yahoo: 0.522 $\pm$ 0.005 -> 0.566 $\pm$ 0.004; Istella: 0.413 $\pm$ 0.001 -> 0.417 $\pm$ 0.002; Web30k: 0.236 $\pm$ 0.00 | Improved |
| Use zero- or mean-imputed privileged features at test time instead of training without privileged features (ablation) | PFD teacher · Yahoo, Istella, Web30k | NDCG@8: Yahoo: 0.329 / 0.357 vs 0.517; Istella: 0.210 / 0.244 vs 0.402; Web30k: 0.177 / 0.181 vs 0.241 | Hurt |
| Reduce the mixing ratio so teacher loss dominates the distillation loss (ablation) | PFD · Yahoo | NDCG@8 | Improved |
| Vary alpha across a broad range while keeping teacher loss effective (ablation) | PFD · Yahoo, Istella, Web30k | NDCG@8: over 2.5\% improvement for Istella and over 2.6\% for Web30k at all alpha <= 0.95 | No clear effect |
| Increase the sparsity of positive labels (ablation) | PFD relative to self-distillation · Yahoo | NDCG@8 | Improved |
| Increase privileged-feature predictive power from low to very high (ablation) | PFD · Yahoo | NDCG@8 | Mixed |
| Use add-to-cart rather than click as the single privileged feature (ablation) | PFD student · Amazon search logs | Purchase NDCG@8: 1.082 -> 1.073 | Hurt |
Closest papers by meaning
- A statistical perspective on distillation (ICML 2021 · similarity 0.84)
- PLD: A Choice-Theoretic List-Wise Knowledge Distillation (NeurIPS 2025 · similarity 0.83)
- The Fundamental Limits of Least-Privilege Learning (ICML 2024 · similarity 0.83)
- Testing Using Privileged Information by Adapting Features With Statistical Dependence (ICCV 2021 · similarity 0.82)
- Enhancing Logits Distillation with Plug&Play Kendall's $\tau$ Ranking Loss (ICML 2025 · similarity 0.82)
- RankSHAP: Shapley Value Based Feature Attributions for Learning to Rank (ICLR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
