Research map / Video / Action recognition
Audio-Visual Contrastive Learning with Temporal Self-Supervision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add nearest-neighbor memory-bank positives (ablation) | Video contrastive learning · UCF101 | 1-NN: 61.5 -> 65.3 | Helped |
| Add memory-bank negatives (ablation) | Video contrastive learning · AugVGG-C | R@1: 65.1 -> 65.5 | Helped |
| Sample negatives from the median-distance neighborhood rather than harder negatives (ablation) | Video contrastive learning · UCF101 | 1-NN: 63.5 -> 65.3 | Helped |
| Weight positives by cross-view similarity rather than uniformly (ablation) | Video contrastive learning · AugVGG-C | R@1: 64.1 -> 65.5 | Helped |
| Include speed prediction (ablation) | Audio temporal self-supervision · ESC50 | linear: 80.4 -> 82.2 | Helped |
| Include direction prediction (ablation) | Audio temporal self-supervision · ESC50 | 1-NN: 56.7 -> 61.0 | Helped |
| Include temporal ordering (ablation) | Audio temporal self-supervision · ESC50 | linear: 80.6 -> 82.2 | Helped |
| Perform speed transformations on the raw waveform rather than resizing the spectrogram (ablation) | Audio temporal self-supervision · AugVGG-C | R@1: 19.3 -> 21.9 | Helped |
| Include audio-video contrastive learning (ablation) | Audio-visual model · AugVGG-C | R@1: 73.8 -> 78.1 | Helped |
| Include video-video contrastive learning (ablation) | Audio-visual model · AugVGG-C | R@1: 69.6 -> 78.1 | Helped |
| Add audio-audio contrastive learning (ablation) | Audio-visual model · UCF101 | 1-NN: 70.7 -> 69.1 | Hurt |
| Include temporal self-supervision (ablation) | Audio-visual model · AugVGG-C | R@1: 78.6 -> 78.1 | No clear effect |
Closest papers by meaning
- Robust Audio-Visual Instance Discrimination (CVPR 2021 · similarity 0.92)
- Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization (NeurIPS 2018 · similarity 0.91)
- Audio-Visual Instance Discrimination with Cross-Modal Agreement (CVPR 2021 · similarity 0.91)
- No More Shortcuts: Realizing the Potential of Temporal Self-Supervision (AAAI 2024 · similarity 0.88)
- CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment (CVPR 2025 · similarity 0.87)
- EquiAV: Leveraging Equivariance for Audio-Visual Contrastive Learning (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.