Research map / Label-efficient and robust learning / Recognition and long-tail
Positive-Congruent Training: Towards Regression-Free Model Updates
Sijie Yan, Yuanjun Xiong, Kaustav Kundu, Shuo Yang, Siqi Deng, Meng Wang, Wei Xia, Stefano Soatto
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply focal distillation with logit matching | ResNet-18 model updates · ILSVRC12 | NFR (%): 6.44 -> 2.35 | Improved |
| Apply focal distillation with KL divergence | ResNet-18 model updates · iNaturalist | NFR (%): 7.77 -> 2.83 | Improved |
| Use ensembles as the old and new models | ResNet-18 model updates · ILSVRC12 | NFR (%): 6.44 -> 1.70 | Improved |
| Apply focal distillation with logit matching | ResNet-18 → ResNet-50 update · ILSVRC12 | NFR (%): 4.89 -> 2.92 | Improved |
| Apply focal distillation with KL divergence | ResNet-18 → DenseNet-161 update · ILSVRC12 | NFR (%): 4.03 -> 2.50 | Improved |
| Use focal distillation | ResNet-18 model updates with more training samples · ILSVRC12 | NFR (%): 3.98 -> 2.85 | Improved |
| Use focal distillation with KL divergence | ResNet-18 model updates with more classes · ILSVRC12 | NFR (%): 8.05 -> 7.07 | Improved |
| Use distillation only on samples correctly classified by the old model (ablation) | Focal Distillation · ILSVRC12 | NFR (%): 6.44 -> 5.25 | Hurt |
| Set the focal weight β to 0, yielding ordinary distillation (ablation) | Focal Distillation · ILSVRC12 | NFR (%): 6.44 -> 3.90 | Improved |
| Increase the focal weight β from 5 to 20 (ablation) | Focal Distillation · ILSVRC12 | NFR (%): 2.35 -> 6.94 | Hurt |
| Use logit matching instead of KL divergence (ablation) | Focal Distillation · ILSVRC12 | NFR (%): 2.50 -> 2.35 | Improved |
| Increase ensemble size (ablation) | Ensemble model updates · ILSVRC12 | NFR | Improved |
Closest papers by meaning
- Mitigating Negative Flips via Margin Preserving Training (AAAI 2026 · similarity 0.88)
- Verification of Neural Networks Against Convolutional Perturbations via Parameterised Kernels (AAAI 2025 · similarity 0.82)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.82)
- Improved Techniques for Training Consistency Models (ICLR 2024 · similarity 0.82)
- Dataset Distillation for Pre-Trained Self-Supervised Vision Models (NeurIPS 2025 · similarity 0.82)
- SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation (ICLR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
