Research map / Label-efficient and robust learning / Knowledge distillation
DnS: Distill-and-Select for Efficient and Accurate Video Indexing and Retrieval
Giorgos Kordopatis-Zilos, Christos Tzelepis, Symeon Papadopoulos, Ioannis Yiannis Kompatsiaris, Ioannis Patras
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use h-attention (with fully-connected layer) instead of l2-attention (ablation) | fine-grained attention student · FIVR-5K ISVR | mAP: 0.787 -> 0.803 | Improved |
| use the differentiable E[sgn(x)] (erf) activation instead of sgn (ablation) | fine-grained binarization student · FIVR-5K ISVR | mAP: 0.776 -> 0.788 | Improved |
| use E[sgn(x)] activation instead of tanh(beta x) (ablation) | fine-grained binarization student · FIVR-5K DSVR | mAP: 0.875 -> 0.879 | Improved |
| increase bits per region vector from 64 to 512 (ablation) | fine-grained binarization student · FIVR-5K DSVR | mAP: 0.845 -> 0.879 | Improved |
| distill from teacher on unlabelled data instead of supervised training on VCDB (ablation) | fine-grained attention student · FIVR-5K DSVR | mAP: 0.821 -> 0.873 | Improved |
| distill on DnS-100K instead of VCDB (ablation) | coarse-grained student · FIVR-5K DSVR | mAP: 0.510 -> 0.634 | Improved |
| increase training data from 25% to 100% of DnS-100K (ablation) | coarse-grained student · FIVR-5K DSVR | mAP: 0.573 -> 0.634 | Improved |
| increase training data from 25% to 100% of DnS-100K (ablation) | fine-grained attention student · FIVR-5K DSVR | mAP: 0.888 -> 0.893 | No clear effect |
| distill from a better teacher (S^f_A(2) instead of T) (ablation) | fine-grained binarization student · FIVR-5K DSVR | mAP: 0.879 -> 0.895 | Improved |
| distill from a better teacher (S^f_A(2) instead of T) (ablation) | coarse-grained student · FIVR-5K DSVR | mAP: 0.634 -> 0.632 | No clear effect |
| binarize region features (32 times smaller) instead of float features | fine-grained binarization student vs teacher · FIVR-5K DSVR | mAP and storage: 0.882 -> 0.879 mAP; 15124 KB -> 63 KB | Improved |
| replace fine-grained teacher by the coarse-grained student | S^c vs T · FIVR-5K DSVR | mAP and time: 0.882 -> 0.634 mAP; 10.10 -> 0.018 s per query | Mixed |
Closest papers by meaning
- Let All Be Whitened: Multi-Teacher Distillation for Efficient Visual Retrieval (AAAI 2024 · similarity 0.87)
- DGL: Dynamic Global-Local Prompt Tuning for Text-Video Retrieval (AAAI 2024 · similarity 0.85)
- Text Proxy: Decomposing Retrieval from a 1-to-N Relationship into N 1-to-1 Relationships for Text-Video Retrieval (AAAI 2025 · similarity 0.85)
- TempMe: Video Temporal Token Merging for Efficient Text-Video Retrieval (ICLR 2025 · similarity 0.85)
- Mitigating Semantic Collapse in Partially Relevant Video Retrieval (NeurIPS 2025 · similarity 0.84)
- From Play to Replay: Composed Video Retrieval for Temporally Fine-Grained Videos (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
