Research map / Label-efficient and robust learning / Self-supervised and contrastive
Putting An End to End-to-End: Gradient-Isolated Learning of Representations
Sindy Löwe, Peter O'Connor, Bastiaan Veeling
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use greedy self-supervised InfoNCE training instead of end-to-end CPC | Audio representations on LibriSpeech · LibriSpeech 100-hour subset | Phone classification accuracy (%): 64.9 -> 62.5 | Hurt |
| Use greedy self-supervised InfoNCE training instead of end-to-end CPC | Audio representations on LibriSpeech · LibriSpeech 100-hour subset | Speaker classification accuracy (%): 99.6 -> 99.4 | No clear effect |
| Use greedy self-supervised InfoNCE training instead of supervised end-to-end training | Audio representations on LibriSpeech · LibriSpeech 100-hour subset | Speaker classification accuracy (%): 98.9 -> 99.4 | Improved |
| Use greedy self-supervised InfoNCE training instead of greedy supervised training | Audio representations on LibriSpeech · LibriSpeech 100-hour subset | Phone classification accuracy (%): 73.4 -> 62.5 | Hurt |
| Train modules iteratively to convergence, freezing each before training the next (ablation) | GIM image classifier · STL-10 test set | Classification accuracy: 81.9% -> 79.8% | Hurt |
| Train modules asynchronously rather than simultaneously | GIM with ResNet-50 · GPU memory consumption during training | Memory reduction: 2.8 | Improved |
| Remove backpropagation through time by blocking gradients between autoregressive time-steps (ablation) | GIM audio model · LibriSpeech phone classification | Phone classification performance | Hurt |
| Remove backpropagation through time by blocking gradients between autoregressive time-steps (ablation) | GIM audio model · LibriSpeech speaker classification | Speaker classification performance | No clear effect |
| Remove the autoregressive module (ablation) | GIM audio model · LibriSpeech phone classification | Phone classification performance | Hurt |
| Remove the autoregressive module (ablation) | GIM audio model · LibriSpeech speaker classification | Speaker classification performance | No clear effect |
Closest papers by meaning
- Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation (ICML 2024 · similarity 0.87)
- Characterizing emergent representations in a space of candidate learning rules for deep networks (NeurIPS 2020 · similarity 0.87)
- Gradients as Features for Deep Representation Learning (ICLR 2020 · similarity 0.86)
- Backpropagation-Free Deep Learning with Recursive Local Representation Alignment (AAAI 2023 · similarity 0.86)
- Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Models (ICLR 2024 · similarity 0.84)
- Dendritic Localized Learning: Toward Biologically Plausible Algorithm (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
