Research map / Probabilistic and scientific modeling / Variational and generative sampling
A Unified Knowledge Distillation Framework for Deep Directed Graphical Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use semi-auxiliary surrogate distillation instead of local distillation | Hierarchical VAE compression · SVHN | FID: 5.93 ± 0.50 -> 4.38 ± 0.05 | Helped |
| Use semi-auxiliary surrogate distillation instead of local distillation | Hierarchical VAE compression · CelebA | FID: 6.23 ± 0.17 -> 5.38 ± 0.10 | Helped |
| Use semi-auxiliary surrogate distillation instead of local distillation | Hierarchical VAE compression · Cifar10 | FID: 6.22 ± 0.05 -> 5.47 ± 0.23 | Helped |
| Distill the student rather than train it from scratch | Hierarchical VAE compression · SVHN | FID: 4.69 ± 0.16 -> 4.38 ± 0.05 | Helped |
| Distill the student rather than train it from scratch | Hierarchical VAE compression · CelebA | FID: 6.10 ± 0.31 -> 5.38 ± 0.10 | Helped |
| Distill the student rather than train it from scratch | Hierarchical VAE compression · Cifar10 | FID: 6.19 ± 0.25 -> 5.47 ± 0.23 | Helped |
| Use the proposed KD loss rather than generative replay | VAE continual learning · CelebA, Female →Male | FID (Old): 5.57(+0.99) -> 4.39(-0.25) | Helped |
| Use the proposed KD loss rather than local distillation | VAE continual learning · CelebA, No Eyeglasses →Eyeglasses | FID (Old): 6.51(+2.06) -> 4.43(-0.02) | Helped |
| Use the proposed KD loss rather than local distillation | VAE continual learning · CelebA, Female →Male | FID (New): 4.25 -> 4.19 | No clear effect |
Closest papers by meaning
- Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-training of Deep Networks (ICLR 2025 · similarity 0.90)
- DA-KD: Difficulty-Aware Knowledge Distillation for Efficient Large Language Models (ICML 2025 · similarity 0.89)
- DistiLLM: Towards Streamlined Distillation for Large Language Models (ICML 2024 · similarity 0.88)
- DeepKD: A Deeply Decoupled and Denoised Knowledge Distillation Trainer (NeurIPS 2025 · similarity 0.88)
- Small Scale Data-Free Knowledge Distillation (CVPR 2024 · similarity 0.88)
- Progressively Knowledge Distillation via Re-parameterizing Diffusion Reverse Process (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.