Research map / Label-efficient and robust learning / Self-supervised and contrastive
Robust Audio-Visual Instance Discrimination
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| down-weight weak audio-visual correspondences during training (ablation) | xID · UCF and HMDB | accuracy: 68.0 -> 69.7; 39.0 -> 40.1 | Helped |
| replace one-hot targets with CCP soft targets (ablation) | xID · UCF and HMDB | accuracy: 68.0 -> 70.3; 39.0 -> 41.5 | Helped |
| combine robust sample weighting with CCP soft targets | xID · UCF and HMDB | accuracy: 68.0 -> 71.6; 39.0 -> 41.9 | Helped |
| use bootstrapped model predictions rather than one-hot targets (ablation) | xID · UCF and HMDB | accuracy: 68.0 -> 69.2; 39.0 -> 40.5 | Helped |
| use within-modal neighbor soft targets (ablation) | xID · UCF and HMDB | accuracy: 68.0 -> 70.5; 39.0 -> 41.2 | Helped |
| use opposite-modality swapped-prediction soft targets (ablation) | xID · UCF and HMDB | accuracy: 68.0 -> 70.0; 39.0 -> 41.3 | Helped |
| use cycle-consistent prediction for soft targets (ablation) | xID · UCF and HMDB | accuracy: 68.0 -> 70.3; 39.0 -> 41.5 | Helped |
| increase the Soft-xID mixing coefficient substantially above 0.65 (ablation) | Soft-xID · UCF and HMDB | transfer performance | Hurt |
| inject more faulty positive audio-video pairs (ablation) | xID · UCF and HMDB | transfer performance | Hurt |
Closest papers by meaning
- Audio-Visual Instance Discrimination with Cross-Modal Agreement (CVPR 2021 · similarity 0.96)
- Audio-Visual Contrastive Learning with Temporal Self-Supervision (AAAI 2023 · similarity 0.92)
- Self-Supervised Audio-Visual Representation Learning with Relaxed Cross-Modal Synchronicity (AAAI 2023 · similarity 0.90)
- Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization (NeurIPS 2018 · similarity 0.89)
- EquiAV: Leveraging Equivariance for Audio-Visual Contrastive Learning (ICML 2024 · similarity 0.87)
- ES3: Evolving Self-Supervised Learning of Robust Audio-Visual Speech Representations (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.