Research map / Label-efficient and robust learning / Knowledge distillation

AugKD: Ingenious Augmentations Empower Knowledge Distillation for Image Super-Resolution

ICLR 2025 · Label-efficient and robust learning › Knowledge distillation · approach: Few-shot and metric distillation

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AugKD, a knowledge distillation method for image super-resolution that uses augmentations to improve teacher-student knowledge transfer, since vanilla KD yields limited gains because teacher outputs are noisy approximations of label images.

Problem

Vanilla knowledge distillation for image super-resolution networks gives only limited improvement because teacher outputs are noisy approximations of high-quality label images.

Approach

Uses ingenious augmentations to empower knowledge distillation for SR, so that the student benefits more from teacher knowledge than vanilla KD allows (details not stated in the provided abstract).

Surprising findings

• Vanilla KD for image super-resolution yields only limited improvements, attributed to teacher outputs being noisy approximations of high-quality label images.

Closest papers by meaning

• Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution (ICLR 2025 · similarity 0.88)
• VkD: Improving Knowledge Distillation using Orthogonal Projections (CVPR 2024 · similarity 0.87)
• Dynamic Contrastive Knowledge Distillation for Efficient Image Restoration (AAAI 2025 · similarity 0.86)
• ADD: Attribution-Driven Data Augmentation Framework for Boosting Image Super-Resolution (CVPR 2025 · similarity 0.85)
• ScaleKD: Strong Vision Transformers Could Be Excellent Teachers (NeurIPS 2024 · similarity 0.85)
• SeD: Semantic-Aware Discriminator for Image Super-Resolution (CVPR 2024 · similarity 0.85)