Research map / Architectures and efficiency / RNNs and normalization
Learning with Retrospection
Xiang Deng, Zhongfei Zhang
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add periodically updated soft labels from past training logits | ResNet-56 · CIFAR-100 | test accuracy: 72.00$\pm$0.16 -> 74.25$\pm$0.29 | Improved |
| Add periodically updated soft labels from past training logits | WRN-16-4 · Tiny ImageNet | test accuracy: 59.48$\pm$0.16 -> 61.22$\pm$0.29 | Improved |
| Add periodically updated soft labels from past training logits | ResNet-18 · CUB | test accuracy: 61.09$\pm$0.49 -> 66.47$\pm$0.37 | Improved |
| Add periodically updated soft labels from past training logits | PreAct ResNet-18 · CIFAR-10 with 80% corrupted labels, last epoch | test accuracy: 27.08$\pm$1.51 -> 72.27$\pm$0.19 | Improved |
| Add periodically updated soft labels from past training logits | ShuffleNetV2 · CIFAR-100 | ECE: 12.60$\pm$0.16 -> 2.87$\pm$0.06 | Improved |
| Increase soft-label update interval to k=100 (ablation) | LWR with PreAct ResNet-18 · CIFAR-100 | test accuracy: 79.73$\pm$0.22 -> 76.32$\pm$0.11 | Hurt |
| Use update interval k=1 instead of k=5 (ablation) | LWR with PreAct ResNet-18 · CIFAR-100 | test accuracy: 79.73$\pm$0.22 -> 79.50$\pm$0.10 | No clear effect |
| Use temperature τ=10 rather than τ=5 (ablation) | LWR with PreAct ResNet-18 and k=10 · CIFAR-100 | test accuracy: 79.27$\pm$0.10 -> 79.60$\pm$0.17 | No clear effect |
| Increase temperature from τ=5 to τ=10 (ablation) | LWR with PreAct ResNet-18 and k=5 · CIFAR-100 | test accuracy: 79.73$\pm$0.22 -> 79.12$\pm$0.03 | Hurt |
| Increase soft-label update interval from k=5 to k=100 (ablation) | LWR with ResNet-56 · CIFAR-100 | test accuracy: 74.25$\pm$0.29 -> 72.98$\pm$0.23 | Hurt |
Closest papers by meaning
- Reducing Flipping Errors in Deep Neural Networks (AAAI 2022 · similarity 0.85)
- Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations (ECCV 2020 · similarity 0.85)
- Adaptive Neural Networks for Efficient Inference (ICML 2017 · similarity 0.85)
- Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers (NeurIPS 2025 · similarity 0.84)
- The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples (NeurIPS 2025 · similarity 0.84)
- Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
