Research map / Label-efficient and robust learning / Knowledge distillation
RD-Suite: A Benchmark for Ranking Distillation
Zhen Qin, Rolf Jagerman, Rama Kumar Pasumarthi, Honglei Zhuang, He Zhang, Aijun Bai, Kai Hui, Le Yan, Xuanhui Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| turn softmax teacher-score transform off for MSE (ablation) | MSE distillation · NQ | MRR@10: 54.87 -> 58.49 | Improved |
| turn softmax teacher-score transform off for MSE (ablation) | MSE distillation · MSMARCO | MRR@10: 40.03 -> 42.06 | Improved |
| turn softmax teacher-score transform off for PairMSE (ablation) | PairMSE · NQ | MRR@10: 55.61 -> 58.34 | Improved |
| turn softmax transform on for Softmax loss (ablation) | Softmax listwise distillation · NQ | MRR@10: 52.67 -> 59.08 | Improved |
| turn softmax transform on for GumbelNDCG (ablation) | GumbelNDCG · NQ | MRR@10: 53.06 -> 57.90 | Improved |
| turn softmax transform on for LambdaLoss (ablation) | LambdaLoss · NQ | MRR@10: 56.52 -> 58.36 | Improved |
| add RankDistil distillation to relevance-only training | BERT ranker · MSMARCO | MRR@10: 40.03 -> 41.29 | Improved |
| add RD distillation to relevance-only training | BERT ranker · MSMARCO | MRR@10: 40.03 -> 40.25 | No clear effect |
| add Softmax listwise distillation to relevance-only training | BERT ranker · NQ | MRR@10: 52.67 -> 59.08 | Improved |
| distill from a weak MSMARCO teacher into NQ | Softmax distillation · NQ transfer | MRR@10: 52.67 -> 56.72 | Improved |
| add RD distillation on tabular Web30K | MLP ranker · Web30K | MRR@10: 27.81 -> 27.44 | No clear effect |
| add Softmax distillation on Web30K | MLP ranker · Web30K | MRR@10: 27.81 -> 29.54 | Improved |
Closest papers by meaning
- A statistical perspective on distillation (ICML 2021 · similarity 0.87)
- Foundations of the Theory of Performance-Based Ranking (CVPR 2025 · similarity 0.85)
- Robust Consensus in Ranking Data Analysis: Definitions, Properties and Computational Issues (ICML 2023 · similarity 0.85)
- Both Supply and Precision: Sample Debias and Ranking Consistency Joint Learning for Large Scale Pre-Ranking System (AAAI 2025 · similarity 0.83)
- RankSHAP: Shapley Value Based Feature Attributions for Learning to Rank (ICLR 2025 · similarity 0.83)
- Stability and Multigroup Fairness in Ranking with Uncertain Predictions (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
