Research map / Image generation / Super-resolution
MTKD: Multi-Teacher Knowledge Distillation for Image Super-Resolution
Yuxuan Jiang, Chen Feng, Fan Zhang, David Bull
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use three teachers (EDSR, RCAN, SwinIR) instead of SwinIR only (ablation) | MTKD with RCAN_lightweight · Urban100 x4 | PSNR: 26.70 -> 27.08 | Improved |
| add EDSR as second teacher alongside SwinIR (ablation) | MTKD with RCAN_lightweight · Urban100 x4 | PSNR: 26.70 -> 26.79 | Improved |
| use DCTSwin blocks instead of Mixer Layer blocks in the aggregation network (ablation) | MTKD knowledge aggregation · Urban100 x4 | PSNR: 26.93 -> 27.08 | Improved |
| add DCT and IDCT modules to the aggregation network (ablation) | MTKD knowledge aggregation · Urban100 x4 | PSNR: 26.89 -> 27.08 | Improved |
| use wavelet-based distillation loss instead of L1 loss (ablation) | MTKD · Urban100 x4 | PSNR: 26.78 -> 27.08 | Improved |
| use wavelet loss over DCT-based loss (ablation) | MTKD · Urban100 x4 | PSNR: 26.98 -> 27.08 | Improved |
| use wavelet loss over high-frequency-only DWT loss (ablation) | MTKD · Urban100 x4 | PSNR: 26.28 -> 27.08 | Improved |
| apply MTKD instead of other KD methods across students and scales | RCAN_lightweight, EDSR_baseline, SwinIR_lightweight · Set5/Set14/BSD100/Urban100 x2 x3 x4 | PSNR: up to 0.46 dB over second best | Improved |
| distill with MTKD into RCAN_lightweight | RCAN_lightweight · Urban100 x4 | PSNR: 27.08 vs 26.82 (full RCAN) | Improved |
| gain over other KD methods depends on teacher-student gap | MTKD · multiple students | PSNR | Mixed |
Closest papers by meaning
- Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution (ICLR 2025 · similarity 0.92)
- AugKD: Ingenious Augmentations Empower Knowledge Distillation for Image Super-Resolution (ICLR 2025 · similarity 0.91)
- Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks (TPAMI 2021 · similarity 0.91)
- Dynamic Contrastive Knowledge Distillation for Efficient Image Restoration (AAAI 2025 · similarity 0.89)
- Multi-Teacher Knowledge Distillation with Reinforcement Learning for Visual Recognition (AAAI 2025 · similarity 0.87)
- All You Need in Knowledge Distillation Is a Tailored Coordinate System (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
