Research map / Label-efficient and robust learning / Knowledge distillation
SDPGO: Efficient Self-Distillation Training Meets Proximal Gradient Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SDPGO, a self-knowledge distillation method that uses proximal gradient optimization to extract key features for single-model training.
Problem
Self-knowledge distillation methods mainly replicate common features in the student model and neglect extracting key features that would enhance student learning.
Approach
Applies proximal gradient optimization within self-distillation so the student model focuses on key features from its own outputs rather than only replicating common features; the abstract is truncated so details are not fully stated.
Experimental setup
Builds on: self-knowledge distillation
Closest papers by meaning
- Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation (CVPR 2021 · similarity 0.90)
- Self-Distillation From the Last Mini-Batch for Consistency Regularization (CVPR 2022 · similarity 0.90)
- Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-training of Deep Networks (ICLR 2025 · similarity 0.89)
- VkD: Improving Knowledge Distillation using Orthogonal Projections (CVPR 2024 · similarity 0.88)
- Small Scale Data-Free Knowledge Distillation (CVPR 2024 · similarity 0.88)
- Boost Self-Supervised Dataset Distillation via Parameterization, Predefined Augmentation, and Approximation (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.