AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Toward Understanding Privileged Features Distillation in Learning-to-Rank

Shuo Yang, sujay sanghavi, Holakou Rahmanian, Jan Bakus, Vishwanathan S. V. N.

NeurIPS 2022 · Label-efficient and robust learning › Knowledge distillation · approach: Few-shot and metric distillation · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use a teacher with regular and privileged features for distillation instead of no distillationRanking model · Yahoo, Istella, Web30kNDCG@8: 0.517 $\pm$ 0.005 (+0.0\%) -> 0.566 $\pm$ 0.004 (+9.5\%); 0.402 $\pm$ 0.001 (+0.0\%) -> 0.417 $\pm$ 0.002 (+3.7\%); 0.24Improved
Use a teacher with both regular and privileged features rather than a privileged-features-only teacherRanking model · Yahoo, Istella, Web30kNDCG@8: 0.557 -> 0.566; 0.380 -> 0.417; 0.205 -> 0.252Mixed
Add click as a privileged feature to PFDAmazon purchase ranking · Amazon search logsPurchase NDCG@8: 1.000 -> 1.082Improved
Distill from position, click, and add-to-cart teachers rather than a single teacherPFD student · Amazon search logsPurchase NDCG@8: +11.2\% over baselineImproved
Use positive query groups rather than all query groups for the data loss (ablation)PFD · Yahoo, Istella, Web30kNDCG@8: Yahoo: 0.522 $\pm$ 0.005 -> 0.566 $\pm$ 0.004; Istella: 0.413 $\pm$ 0.001 -> 0.417 $\pm$ 0.002; Web30k: 0.236 $\pm$ 0.00Improved
Use zero- or mean-imputed privileged features at test time instead of training without privileged features (ablation)PFD teacher · Yahoo, Istella, Web30kNDCG@8: Yahoo: 0.329 / 0.357 vs 0.517; Istella: 0.210 / 0.244 vs 0.402; Web30k: 0.177 / 0.181 vs 0.241Hurt
Reduce the mixing ratio so teacher loss dominates the distillation loss (ablation)PFD · YahooNDCG@8Improved
Vary alpha across a broad range while keeping teacher loss effective (ablation)PFD · Yahoo, Istella, Web30kNDCG@8: over 2.5\% improvement for Istella and over 2.6\% for Web30k at all alpha <= 0.95No clear effect
Increase the sparsity of positive labels (ablation)PFD relative to self-distillation · YahooNDCG@8Improved
Increase privileged-feature predictive power from low to very high (ablation)PFD · YahooNDCG@8Mixed
Use add-to-cart rather than click as the single privileged feature (ablation)PFD student · Amazon search logsPurchase NDCG@8: 1.082 -> 1.073Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy