Research map / Architectures and efficiency / RNNs and normalization
Improving Anytime Prediction with Parallel Cascaded Networks and a Temporal-Difference Loss
Michael Louis Iuzzolino, Michael Curtis Mozer, Samy Bengio
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TD(λ) with λ below 1 rather than TD(1) (ablation) | Cascaded ResNet · CIFAR-100 | accuracy: 63.20 ± 0.14 -> 67.48 ± 0.14 | Improved |
| Use TD(λ) with λ below 1 rather than TD(1) (ablation) | Cascaded ResNet · CIFAR-10 | accuracy: 88.75 ± 0.42 -> 91.65 ± 0.08 | Improved |
| Use TD(λ) with λ below 1 rather than TD(1) (ablation) | Cascaded ResNet · TinyImageNet | accuracy: 49.86 ± 0.15 -> 52.25 ± 0.07 | Improved |
| Add TD(0) training instead of standard cross-entropy training (ablation) | Cascaded ResNet · CIFAR-100 | accuracy: 65.56 ± 0.06 -> 67.48 ± 0.14 | Improved |
| Add TD(0) training instead of standard cross-entropy training (ablation) | Cascaded ResNet · CIFAR-10 | accuracy: 91.91 ± 0.08 -> 91.22 ± 0.18 | Hurt |
| Add TD(0) training instead of standard cross-entropy training (ablation) | Cascaded ResNet · TinyImageNet | accuracy: 52.46 ± 0.06 -> 50.74 ± 0.11 | Hurt |
| Use the cascaded architecture with TD training rather than the serial SDN-style model | Anytime image classification · CIFAR-10, CIFAR-100, TinyImageNet | speed-accuracy trade-off | Improved |
| Use multi-head instead of shared-head readouts (ablation) | Serial anytime model · CIFAR-10, CIFAR-100, TinyImageNet | speed-accuracy trade-off | Improved |
| Use multi-head instead of shared-head readouts (ablation) | Cascaded anytime model · CIFAR-10, CIFAR-100, TinyImageNet | speed-accuracy trade-off | Hurt |
| Train with TD rather than standard cross-entropy (ablation) | Cascaded model · CIFAR-10 prototypicality | Spearman correlation with negative selection latency: 0.140 -> 0.352; 0.326 -> 0.489; 0.153 -> 0.142 | Mixed |
| Train CascadedTD instead of CascadedCE (ablation) | Cascaded ResNet · CIFAR-10 persistent lossy noise | asymptotic accuracy (%): 83.75 ± 0.10 -> 87.31 ± 0.04; 82.73 ± 0.09 -> 89.76 ± 0.05; 84.56 ± 0.05 -> 87.67 ± 0.08; 85.40 ± 0.07 -> 88.19 ± 0.10 | Improved |
| Train CascadedTD instead of CascadedCE (ablation) | Cascaded ResNet · CIFAR-10 transient lossy noise | drop in integrated performance: 0.66 ± 0.05 -> 0.00 ± 0.01; 8.25 ± 0.72 -> 0.93 ± 0.15; 0.87 ± 0.07 -> 0.00 ± 0.01; 0.53 ± 0.05 -> 0.18 ± 0.02 | Improved |
Closest papers by meaning
- Recurrent neural network dynamical systems for biological vision (NeurIPS 2024 · similarity 0.85)
- Emergent mechanisms for long timescales depend on training curriculum and affect performance in memory tasks (ICLR 2024 · similarity 0.85)
- Feature Learning Dynamics in Infinite-Depth Neural Networks (NeurIPS 2026 · similarity 0.85)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.84)
- Training a Predictive Coding Network on ImageNet using Equilibrium Propagation (NeurIPS 2026 · similarity 0.84)
- Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
