Research map / Label-efficient and robust learning / Noisy labels
Random Label Prediction Heads for Studying Memorization in Deep Neural Networks
Marlon Becker, Jonas Konrad, Luis Garcia Rodriguez, Benjamin Risse
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RLP-regularization (lambda=1e4) on final-layer RLP-head (ablation) | ViT-B/32 · ImageNet | test accuracy: +1.5% to 68.5% | Improved |
| add RLP-regularization (ablation) | WideResNet-16-4 · CIFAR-100 | test accuracy | Hurt |
| add RLP-regularization (ablation) | ViT · CIFAR-100 | test accuracy | Hurt |
| add RLP-regularization when training on small subsets of ImageNet (ablation) | ViT-B/32 · ImageNet subsets | test accuracy | No clear effect |
| add RLP-regularization with injected label noise (ablation) | ViT-B/32 · ImageNet with label noise | test accuracy | Improved |
| add RLP-regularization using all layers' RLP-heads (ablation) | ViT-B/32 · ImageNet | test accuracy | No clear effect |
| add dropout, weight decay or label smoothing (ablation) | WRN16-4 · CIFAR-100 | random label accuracy | Improved |
| regularize only final-layer RLP-head (ablation) | ViT-B/32 · ImageNet | random label accuracy in layers 2-6 | Mixed |
Closest papers by meaning
- Exploring Memorization in Adversarial Training (ICLR 2022 · similarity 0.90)
- The Curious Case of Benign Memorization (ICLR 2023 · similarity 0.89)
- Unveiling Memorization–Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise (NeurIPS 2026 · similarity 0.89)
- Memorization in Graph Neural Networks (NeurIPS 2025 · similarity 0.87)
- Towards Memorization Estimation: Fast, Formal and Free (ICML 2025 · similarity 0.86)
- To Grok or not to Grok: Disentangling Generalization and Memorization on Corrupted Algorithmic Datasets (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
