Research map / Label-efficient and robust learning / Knowledge distillation
Learning Student Networks in the Wild
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| select unlabeled images with low teacher-prediction noise and apply noisy-label adaptation | ResNet-18 student distilled from ResNet-34 · CIFAR-10 | accuracy: 91.25% -> 94.02% | Helped |
| use unlabeled data with DFND rather than generated data in DFAD | ResNet-18 student distilled from ResNet-34 · CIFAR-100 | accuracy: 67.70% -> 76.35% | Helped |
| use DFND with wild unlabeled data instead of PU-compression with some original data | ResNet-18 student distilled from ResNet-34 · ImageNet | top-1 accuracy: 61.92% -> 61.75% | Hurt |
| use DFND with wild unlabeled data instead of PU-compression with some original data | ResNet-18 student distilled from ResNet-34 · ImageNet | top-5 accuracy: 86.00% -> 85.93% | Hurt |
| use DFND on unlabeled ImageNet instead of generated data in DFAD | FCN MobileNetV2 student · NYUv2 | mIOU: 0.364 -> 0.378 | Helped |
| add a noisy adaptation matrix initialized with Eq. 14 and learn it (ablation) | DFND on CIFAR-100 · CIFAR-100 | accuracy: 75.73% -> 76.35% | Helped |
| learn Q instead of keeping Q fixed, with identity initialization (ablation) | DFND on CIFAR-100 · CIFAR-100 | accuracy: 75.60% -> 76.07% | Helped |
| learn Q instead of keeping Q fixed, with Eq. 14 initialization (ablation) | DFND on CIFAR-100 · CIFAR-100 | accuracy: 75.85% -> 76.35% | Helped |
| initialize Q using Eq. 14 rather than identity while keeping Q fixed (ablation) | DFND on CIFAR-100 · CIFAR-100 | accuracy: 75.60% -> 75.85% | Helped |
| initialize learnable Q using Eq. 14 rather than identity (ablation) | DFND on CIFAR-100 · CIFAR-100 | accuracy: 76.07% -> 76.35% | Helped |
Closest papers by meaning
- Hybrid Data-Free Knowledge Distillation (AAAI 2025 · similarity 0.91)
- Small Scale Data-Free Knowledge Distillation (CVPR 2024 · similarity 0.90)
- Few Sample Knowledge Distillation for Efficient Network Compression (CVPR 2020 · similarity 0.89)
- Improved Knowledge Distillation via Teacher Assistant (AAAI 2020 · similarity 0.89)
- Learning Student Networks with Few Data (AAAI 2020 · similarity 0.89)
- Impartial Adversarial Distillation: Addressing Biased Data-Free Knowledge Distillation via Adaptive Constrained Optimization (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.