Research map / Label-efficient and robust learning / Knowledge distillation
Amalgamating Knowledge towards Comprehensive Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use knowledge amalgamation with joint parameter learning rather than the ensemble | AlexNet student · Stanford Dogs | accuracy: 43.5% -> 45.3% | Helped |
| use joint parameter learning rather than layer-wise learning alone (ablation) | AlexNet student · Stanford Dogs | accuracy: 38.4% -> 45.3% | Helped |
| add the Feature Adaptation Module (ablation) | knowledge amalgamation student · Stanford Dogs | accuracy: 25.3% -> 45.3% | Helped |
| add the Feature Adaptation Module (ablation) | knowledge amalgamation student · CUB-200-2011 | accuracy: 33.9% -> 42.6% | Helped |
| add the Feature Adaptation Module (ablation) | knowledge amalgamation student · FGVC-Aircraft | accuracy: 39.7% -> 49.4% | Helped |
| add the Feature Adaptation Module (ablation) | knowledge amalgamation student · Cars | accuracy: 30.7% -> 40.6% | Helped |
| add layer-wise parameter learning before joint learning (ablation) | AlexNet student · Stanford Dogs | test error: 60.1% | Helped |
| increase the amalgamated teacher count from two to four | DFA student · Stanford Dogs part1 | accuracy: 68.7% -> 69.9% -> 70.3% | Helped |
| use DFA instead of IFA (ablation) | three-teacher student · Stanford Dogs part1 | accuracy: 69.6% -> 69.9% | Helped |
| use DFA instead of IFA (ablation) | three-teacher student · Stanford Dogs part2 | accuracy: 69.1% -> 67.7% | Hurt |
| use DFA instead of IFA (ablation) | four-teacher student · Stanford Dogs part1 | accuracy: 69.9% -> 70.3% | Helped |
| use DFA instead of IFA (ablation) | four-teacher student · Stanford Dogs part4 | accuracy: 67.9% -> 67.3% | Hurt |
Closest papers by meaning
- Distilling Cross-Task Knowledge via Relationship Matching (CVPR 2020 · similarity 0.87)
- Amalgamating Multi-Task Models with Heterogeneous Architectures (AAAI 2024 · similarity 0.87)
- Data-Free Knowledge Amalgamation via Group-Stack Dual-GAN (CVPR 2020 · similarity 0.86)
- Semi-Supervised Knowledge Amalgamation for Sequence Classification (AAAI 2021 · similarity 0.86)
- Less or More From Teacher: Exploiting Trilateral Geometry For Knowledge Distillation (ICLR 2024 · similarity 0.84)
- Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.