AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

Putting An End to End-to-End: Gradient-Isolated Learning of Representations

Sindy Löwe, Peter O'Connor, Bastiaan Veeling

NeurIPS 2019 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Self-supervision in supervised pipelines · 79 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use greedy self-supervised InfoNCE training instead of end-to-end CPCAudio representations on LibriSpeech · LibriSpeech 100-hour subsetPhone classification accuracy (%): 64.9 -> 62.5Hurt
Use greedy self-supervised InfoNCE training instead of end-to-end CPCAudio representations on LibriSpeech · LibriSpeech 100-hour subsetSpeaker classification accuracy (%): 99.6 -> 99.4No clear effect
Use greedy self-supervised InfoNCE training instead of supervised end-to-end trainingAudio representations on LibriSpeech · LibriSpeech 100-hour subsetSpeaker classification accuracy (%): 98.9 -> 99.4Improved
Use greedy self-supervised InfoNCE training instead of greedy supervised trainingAudio representations on LibriSpeech · LibriSpeech 100-hour subsetPhone classification accuracy (%): 73.4 -> 62.5Hurt
Train modules iteratively to convergence, freezing each before training the next (ablation)GIM image classifier · STL-10 test setClassification accuracy: 81.9% -> 79.8%Hurt
Train modules asynchronously rather than simultaneouslyGIM with ResNet-50 · GPU memory consumption during trainingMemory reduction: 2.8Improved
Remove backpropagation through time by blocking gradients between autoregressive time-steps (ablation)GIM audio model · LibriSpeech phone classificationPhone classification performanceHurt
Remove backpropagation through time by blocking gradients between autoregressive time-steps (ablation)GIM audio model · LibriSpeech speaker classificationSpeaker classification performanceNo clear effect
Remove the autoregressive module (ablation)GIM audio model · LibriSpeech phone classificationPhone classification performanceHurt
Remove the autoregressive module (ablation)GIM audio model · LibriSpeech speaker classificationSpeaker classification performanceNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy