Research map / Label-efficient and robust learning / Knowledge distillation
When the Teacher Has More Bits: Self-Teacher Latent Distillation for Learned Image Compression
Abdellah El Mennaoui, Joseph Meehan, Jean-Luc Dugelay, Ghalia Hemrit
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add self-teacher VNLD distillation | HPCM · Tecnick BD-rate (LPIPS) | BD-rate: 0 -> -1.80 | Improved |
| use variance normalization in the latent distillation loss (ablation) | VNLD (HPCM) · HPCM BD-rate (LPIPS) | BD-rate: -0.70 -> -1.60 | Improved |
| use variance normalization (ablation) | VNLD (HPCM) · HPCM BD-rate (PSNR) | BD-rate: -1.03 -> -1.12 | Improved |
| add VNLD vs MSE on BMS | BMS · Kodak BD-rate (MS-SSIM) | BD-rate: +0.4 (MSE) -> -0.5 | Improved |
| use a much higher-rate teacher (lambda 2) instead of adjacent-rate teachers (ablation) | BMS student lambda 0.0018 · BMS RD loss | RD loss: 0.3691 -> 0.3613 | Improved |
| use CRD distillation under frozen decoder | SegPIC · CLIC Pro BD-rate (PSNR) | BD-rate: 0 -> +4.90 | Hurt |
| use VNLD under frozen decoder | SegPIC · Kodak/CLIC Pro/JPEG AI BD-rate | BD-rate: -0.47, -1.60, -1.14 (PSNR) | Improved |
| distillation weight beta above 5e-3 or below 5e-4 (ablation) | VNLD · Kodak (BMS lvl6) | RD performance | Hurt |
| hierarchical or multi-target teacher supervision (ablation) | BMS · Kodak | RD performance | No clear effect |
| domain-specialize with VNLD vs RD-only fine-tuning | SegPIC · Selfie BD-rate (PSNR) | BD-rate: -4.21 | Improved |
| domain-specialize with VNLD on out-of-domain Kodak | SegPIC · Kodak BD-rate (PSNR) | BD-rate: +0.5 | Hurt |
| VNLD vs MSE on HPCM Tecnick PSNR | HPCM · Tecnick BD-rate (PSNR) | BD-rate: -0.91 (MSE) -> -0.65 | Hurt |
Closest papers by meaning
- Knowledge Distillation for Learned Image Compression (ICCV 2025 · similarity 0.92)
- Knowledge distillation via softmax regression representation learning (ICLR 2021 · similarity 0.88)
- Efficient Learned Image Compression without Entropy Coding (ICML 2026 · similarity 0.88)
- Few-Shot Domain Adaptation for Learned Image Compression (AAAI 2025 · similarity 0.87)
- On Disentangled Training for Nonlinear Transform in Learned Image Compression (ICLR 2025 · similarity 0.87)
- Towards Backward-Compatible Continual Learning of Image Compression (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
