Research map / Label-efficient and robust learning / Knowledge distillation
Data-free Knowledge Distillation for Fine-grained Visual Categorization
Renrong Shao, Wei Zhang, Jianhua Yin, Jun Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SFCL contrastive loss to KL baseline (ablation) | data-free KD, ResNet-18 student · Aircraft | accuracy (%): 60.30 -> 63.37 | Improved |
| add MHAD mixed high-order attention distillation to KL baseline (ablation) | data-free KD, ResNet-18 student · Cars196 | accuracy (%): 64.80 -> 69.92 | Improved |
| add SFCL on top of MHAD (ablation) | data-free KD, ResNet-18 student · Cars196 | accuracy (%): 69.92 -> 71.89 | Improved |
| add spatial attention to generator (lambda 0 -> 5e-2) (ablation) | data-free KD · Cars196 | accuracy (%): 69.24 -> 71.89 | Improved |
| increase attention weight lambda from 5e-2 to 9e-2 (ablation) | data-free KD · Cars196 | accuracy (%): 71.89 -> 70.36 | Hurt |
| use 3-order attention distillation instead of 1-order (ablation) | MHAD · Aircraft | accuracy (%): 64.31 -> 65.76 | Improved |
| use 2-order attention distillation instead of 3-order (ablation) | MHAD · Aircraft | accuracy (%): 65.76 -> 63.12 | Hurt |
| use BN-regularization prior (ADI) versus no prior (DFAD) | data-free KD · Aircraft | accuracy (%): 49.51 -> 58.14 | Improved |
Closest papers by meaning
- LLM-assisted Entropy-based Adaptive Distillation for Unsupervised Fine-grained Visual Representation Learning (ICCV 2025 · similarity 0.89)
- Part-level Semantic-guided Contrastive Learning for Fine-grained Visual Classification (ICLR 2026 · similarity 0.89)
- FD²: A Dedicated Framework for Fine-Grained Dataset Distillation (ECCV 2026 · similarity 0.89)
- Democratizing Fine-grained Visual Recognition with Large Language Models (ICLR 2024 · similarity 0.88)
- Fine-R1: Make Multi-modal LLMs Excel in Fine-Grained Visual Recognition by Chain-of-Thought Reasoning (ICLR 2026 · similarity 0.87)
- Novel Class Discovery for Ultra-Fine-Grained Visual Categorization (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
