Research map / Label-efficient and robust learning / Knowledge distillation
Unifying Heterogeneous Classifiers With Distillation
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed cross-entropy or matrix-factorization relation instead of assigning zero probabilities to unclassified classes | Unifying heterogeneous classifiers · ImageNet, LSUN, and Places365 random classes | accuracy: 9-15% | Helped |
| Balance estimated soft labels by inverse class frequency (ablation) | CE and matrix-factorization UHC methods · Random classes | accuracy: less than 1.5% | Helped |
| Balance soft labels (ablation) | UHC methods · Completely overlapping classes | accuracy | Hurt |
| Use direct backpropagation instead of estimating q before training (ablation) | Nonconvex MF-P and MF-LV methods · Random classes | accuracy | Helped |
| Use direct backpropagation instead of estimating q before training (ablation) | Convex CE and MF-LF methods · Random classes | accuracy | No clear effect |
| Increase the size of the unlabelled transfer set (ablation) | UHC methods · ImageNet random classes | accuracy | Helped |
| Vary the temperature used to smooth classifier outputs (ablation) | SD and SD-BS · ImageNet random classes | accuracy | Mixed |
| Use more accurate source classifiers (ablation) | UHC methods · ImageNet random classes | accuracy | Helped |
| Replace MF-LF-BS with MF-P-BP under lower source-classifier accuracy (ablation) | UHC methods · ImageNet random classes | accuracy | Hurt |
Closest papers by meaning
- A Unified Framework for Heterogeneous Semi-supervised Learning (CVPR 2025 · similarity 0.86)
- Debiased Distillation for Consistency Regularization (AAAI 2025 · similarity 0.85)
- A Label is Worth A Thousand Images in Dataset Distillation (NeurIPS 2024 · similarity 0.84)
- Scaled Decoupled Distillation (CVPR 2024 · similarity 0.84)
- Towards Universal Dataset Distillation via Task-Driven Diffusion (CVPR 2025 · similarity 0.84)
- Scaling Up Semi-supervised Learning with Unconstrained Unlabelled Data (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.