Research map / Label-efficient and robust learning / Noisy labels
Transfer and Marginalize: Explaining Away Label Noise with Privileged Information
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add privileged information via TRAM (single-step, weight-shared, approximate marginal head) | MobileNet on CIFAR-10H · CIFAR-10H | NLL: 1.058 -> 0.980 | Helped |
| add privileged information via TRAM | ResNet-50 on relabelled ImageNet · ImageNet validation | accuracy: 71.68 -> 72.54 | Helped |
| TRAM marginalization approximates full MC marginalization | TRAM vs full marginalization on CIFAR-10H · CIFAR-10H | accuracy: 70.06 vs 70.34 | No clear effect |
| mean imputation of PI at test time | MobileNet on CIFAR-10H · CIFAR-10H | NLL: 1.058 -> 0.963 | Helped |
| imputation of PI (zero or mean) at test time | ResNet-50 on relabelled ImageNet · ImageNet validation | accuracy: zero 63.47, mean 65.07 vs No PI 71.68 | Hurt |
| heteroscedastic output head (Het-TRAM) over TRAM (ablation) | TRAM on CIFAR-10H · CIFAR-10H | NLL: 0.980 -> 0.972 | Helped |
| heteroscedastic output head (Het-TRAM) over TRAM (ablation) | TRAM on ImageNet · ImageNet validation | accuracy: 72.54 -> 72.81 | Helped |
| distillation from PI-trained teacher combined with TRAM (Distilled-TRAM) (ablation) | TRAM on ImageNet · ImageNet validation | NLL: 1.225 -> 1.154 | Helped |
| distillation from PI-trained teacher (without TRAM weight sharing, [cite] style) | distillation on CIFAR-10H · CIFAR-10H | NLL: 1.118 (Distillation No PI) vs 1.122 ([cite]) | No clear effect |
| remove label-probability PI feature (keep only model ID) (ablation) | TRAM on ImageNet · ImageNet validation | accuracy: 72.54 -> 72.01 | Hurt |
| reduce network capacity to 1/4 of ResNet-50 parameters (ablation) | TRAM on ImageNet · ImageNet validation (underfitting regime) | NLL: 1.7171 No PI vs 1.7203 TRAM | No clear effect |
| add privileged information via TRAM | Universal Sentence Encoder on Civil Comments Identities · Civil Comments Identities test | NLL: 0.0849 -> 0.0638 | Helped |
Closest papers by meaning
- When does Privileged information Explain Away Label Noise? (ICML 2023 · similarity 0.89)
- Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks (ICLR 2024 · similarity 0.86)
- Understanding Instance-Level Label Noise: Disparate Impacts and Treatments (ICML 2021 · similarity 0.86)
- Toward Robustness against Label Noise in Training Deep Discriminative Neural Networks (NeurIPS 2017 · similarity 0.86)
- Label Noise: Ignorance Is Bliss (NeurIPS 2024 · similarity 0.84)
- Learning with Structural Labels for Learning with Noisy Labels (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.