Research map / Label-efficient and robust learning / Self-supervised and contrastive
On Compositions of Transformations in Contrastive Self-Supervised Learning
Mandela Patrick, Yuki M. Asano, Polina Kuznetsova, Ruth Fong, João F. Henriques, Geoffrey Zweig, Andrea Vedaldi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add audio modality (cross-modal instead of V-only SimCLR-like) (ablation) | GDT · HMDB-51 | finetune accuracy: 44.6 -> 52.4 | Improved |
| add time-shift and time-reversal invariance in visual-only setting (ablation) | SimCLR-like video · HMDB-51 | finetune accuracy: 44.6 -> 37.8 | Hurt |
| add TS and TR invariance in cross-modal setting (ablation) | GDT cross-modal · HMDB-51 | finetune accuracy: 52.4 -> 59.9 | Improved |
| switch TS to distinctiveness (keep TR invariant) (ablation) | GDT cross-modal · HMDB-51 | finetune accuracy: 59.9 -> 61.4 | Improved |
| make both TS and TR distinctive (ablation) | GDT cross-modal · HMDB-51 | finetune accuracy: 61.4 -> 57.2 | Hurt |
| make both TS and TR distinctive (ablation) | GDT cross-modal · HMDB-51 | frozen retrieval R@1: 27.1 -> 20.5 | Hurt |
| learn invariance rather than ignoring TS (ablation) | GDT cross-modal · HMDB-51 | finetune accuracy: 57.4 -> 59.9 | Improved |
| cross-modal instead of within-modal contrast (ablation) | GDT · HMDB-51 | finetune accuracy: 37.8 -> 52.4 | Improved |
| cross-modal instead of within-modal contrast (ablation) | GDT · UCF-101 | finetune accuracy: 76.4 -> 87.6 | Improved |
| video-text cross-modal instead of SimCLR visual only (ablation) | GDT · HMDB-51 (HowTo100M) | accuracy: 36.1 -> 59.2 | Improved |
| add TS distinctiveness and TR invariance to video-text (ablation) | GDT video-text · HMDB-51 (HowTo100M) | accuracy: 59.2 -> 64.4 | Improved |
| add TS+TR invariance in visual-only setting (ablation) | SimCLR-like video · HMDB-51 | frozen retrieval R@1: 11.8 -> 13.9 | Improved |
Closest papers by meaning
- Improving Transformation Invariance in Contrastive Representation Learning (ICLR 2021 · similarity 0.90)
- What Should Not Be Contrastive in Contrastive Learning (ICLR 2021 · similarity 0.90)
- Amortised Invariance Learning for Contrastive Self-Supervision (ICLR 2023 · similarity 0.89)
- On the duality between contrastive and non-contrastive self-supervised learning (ICLR 2023 · similarity 0.88)
- $\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs (ICLR 2025 · similarity 0.87)
- Your contrastive learning problem is secretly a distribution alignment problem (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
