AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Heavy Labels Out! Dataset Distillation with Label Space Lightening

Ruonan Yu, Songhua Liu, Zigeng Chen, Jingwen Ye, Xinchao Wang

ICCV 2025 · Label-efficient and robust learning › Knowledge distillation · approach: Patient consistent distillation · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add multi-weak-teacher guidance to linear-probe CLIP projector (ablation)HeLlO projector · ImageNet-1K IPC 10accuracy: 28.2 -> 30.1Improved
add LoRA-like knowledge transfer (ablation)HeLlO projector · ImageNet-1K IPC 10accuracy: 30.1 -> 43.5Improved
add text-embedding-based initialization (ablation)HeLlO projector · ImageNet-1K IPC 10accuracy: 43.5 -> 43.6No clear effect
add image update with projector (ablation)HeLlO · ImageNet-1K IPC 10accuracy: 43.6 -> 43.7Improved
increase LoRA learnable parameters from 0.3M/0.4M to 0.6M/0.8M (ablation)HeLlO · ImageNet-100 IPC 10accuracy: 45.4 -> 48.9Improved
increase LoRA learnable parameters from 0.6M/0.8M to 1.2M/1.6M (ablation)HeLlO · ImageNet-100 IPC 10accuracy: 48.9 -> 49.8No clear effect
use later-stage teachers (51-91) instead of early (1-41) (ablation)HeLlO · ImageNet-100 IPC 10accuracy: 48.9 -> 41.5Hurt
use later-stage teachers (51-91) instead of mid (11-51) (ablation)HeLlO · ImageNet-1K IPC 10accuracy: 43.7 -> 39.7Hurt
replace RDED soft labels with HeLlO projector at IPC 10HeLlO vs RDED · ImageNet-100accuracy: 36.0 -> 48.9Improved
replace RDED with HeLlO at IPC 50HeLlO vs RDED · ImageNet-1Kaccuracy: 56.5 -> 52.2Hurt
replace RDED with HeLlO on Swin-V2-TinyHeLlO vs RDED · ImageNet-1K IPC 10accuracy: 17.8 -> 29.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy