Research map / Vision-language and multimodal / Cross-modal learning
Learning Deep Representations with Probabilistic Knowledge Transfer
Nikolaos Passalis, Anastasios Tefas
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| transfer knowledge with PKT instead of plain student training | Small CIFAR10 student · CIFAR10 retrieval (ResNet-18 teacher) | mAP: 38.96 -> 51.19 | Improved |
| transfer with PKT instead of distillation | Small CIFAR10 student · CIFAR10 retrieval (ResNet-18 teacher) | mAP: 40.13 -> 51.19 | Improved |
| use hint-based transfer with random projections instead of no transfer | Small CIFAR10 student · CIFAR10 retrieval (ResNet-18 teacher) | mAP: 38.96 -> 21.40 | Hurt |
| use plain distillation on the student | Small CIFAR10 student · CIFAR10 retrieval (ResNet-18 teacher) | mAP: 38.96 -> 40.13 | No clear effect |
| transfer from CS-LBP with PKT instead of hint-based transfer | CNN student · YouTube Faces retrieval (KT from CS-LBP) | mAP: 52.31 -> 54.84 | Improved |
| transfer from CS-LBP with PKT instead of using LBP features | CNN student · YouTube Faces retrieval (KT from CS-LBP) | mAP: 46.38 -> 54.84 | Improved |
| add supervised divergence term (weight 0.001) to PKT (ablation) | PKT · YouTube Faces retrieval (KT from CS-LBP) | mAP: 54.84 -> 70.11 | Improved |
| transfer from HoG with PKT instead of hint-based transfer | CNN student · SUN Attribute retrieval | mAP: 16.40 -> 26.84 | Improved |
| PKT-transferred student from HoG versus the HoG teacher | CNN student · SUN Attribute retrieval | mAP: 32.06 -> 26.84 | Hurt |
| transfer from textual attributes instead of HoG with PKT | CNN student · SUN Attribute retrieval | mAP: 26.84 -> 47.26 | Improved |
| transfer from textual attributes instead of HoG with hint-based transfer | CNN student · SUN Attribute retrieval | mAP: 16.40 -> 26.08 | Improved |
| initialize detector with PKT instead of Darknet ImageNet weights | Tiny YOLO · VOC2007 detection (Tiny YOLO) | mAP: 38.02 -> 44.14 | Improved |
Closest papers by meaning
- Big Transfer (BiT): General Visual Representation Learning (ECCV 2020 · similarity 0.85)
- Transferring Vision-Language Models for Visual Recognition: A Classifier Perspective (IJCV 2023 · similarity 0.85)
- Variational Information Distillation for Knowledge Transfer (CVPR 2019 · similarity 0.84)
- A Probabilistic Approach to Knowledge Translation (AAAI 2016 · similarity 0.84)
- Diffusion Model as Representation Learner (ICCV 2023 · similarity 0.84)
- Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
