Research map / Label-efficient and robust learning / Knowledge distillation
Distilling Knowledge From a Deep Pose Regressor Network
Muhamad Risqi U. Saputra, Pedro P. B. de Gusmao, Yasin Almalioglu, Andrew Markham, Niki Trigoni
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add hint training (HT) to the student (ablation) | Student 72.88%, student loss · KITTI ablation | ATE (m): 80.842 -> 65.251 | Improved |
| replace HT with attentive hint training (AHT) (ablation) | Student 72.88%, student loss · KITTI ablation | ATE (m): 65.251 -> 52.320 | Improved |
| replace student loss with attentive imitation loss (AIL) under HT (ablation) | Student 72.88%, HT · KITTI ablation | ATE (m): 65.251 -> 36.459 | Improved |
| add both AHT and AIL versus HT with student loss (ablation) | Student 72.88% · KITTI ablation | ATE (m): 65.251 -> 32.259 | Improved |
| add hint training (HT) to the student (ablation) | Student 79.69%, student loss · KITTI ablation | ATE (m): 68.350 -> 53.661 | Improved |
| replace student loss with AIL under HT (ablation) | Student 79.69%, HT · KITTI ablation | ATE (m): 53.661 -> 29.751 | Improved |
| replace HT with AHT under student loss (ablation) | Student 79.69% · KITTI ablation | ATE (m): 53.661 -> 37.645 | Improved |
| add both AHT and AIL versus HT with student loss (ablation) | Student 79.69% · KITTI ablation | ATE (m): 53.661 -> 25.857 | Improved |
| use AHT instead of HT for hint transfer (ablation) | Student 72.88% · KITTI ablation | hint reconstruction error: 0.0252 -> 0.0166 | Improved |
| add hint training to student with no HT (ablation) | Student 72.88% · KITTI ablation | hint reconstruction error: 0.6242 -> 0.0252 | Improved |
| use AIL+AHT distillation instead of supervised-only student at 92.95% reduction | Student d_rate 92.95% · KITTI Seq 09-10 | RMS ATE: 71.7517 -> 29.0294 | Improved |
| use proposed distillation instead of KD adapted to regression | Student d_rate 92.95% · KITTI Seq 09-10 | RMS ATE: 165.2182 -> 29.0294 | Improved |
Closest papers by meaning
- Online Knowledge Distillation for Efficient Pose Estimation (ICCV 2021 · similarity 0.86)
- Geometric Loss Functions for Camera Pose Regression With Deep Learning (CVPR 2017 · similarity 0.86)
- Generalizing to the Open World: Deep Visual Odometry With Online Adaptation (CVPR 2021 · similarity 0.86)
- From Variance to Veracity: Unbundling and Mitigating Gradient Variance in Differentiable Bundle Adjustment Layers (CVPR 2024 · similarity 0.84)
- ScaleKD: Strong Vision Transformers Could Be Excellent Teachers (NeurIPS 2024 · similarity 0.83)
- Neural Refinement for Absolute Pose Regression with Feature Synthesis (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
