Research map / Label-efficient and robust learning / Self-supervised and contrastive
AudioMosaic: Contrastive Masked Audio Representation Learning
Hanxun Huang, Qizhou Wang, Xingjun Ma, Cihang Xie, Christopher Leckie, Sarah Monazam Erfani
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use contrastive time-frequency masked pre-training instead of Audio-MAE reconstruction | ViT-B/16 audio encoder · AS-20K linear probing | mAP: 18.3 -> 29.4 | Improved |
| AudioMosaic vs SSLAM under fine-tuning | AudioMosaic · AS-20K | mAP: 40.9 -> 42.5 | Improved |
| AudioMosaic vs SSLAM under fine-tuning | AudioMosaic · AS-2M | mAP: 50.2 -> 50.2 | No clear effect |
| aggregate layers with attentive probe versus best single layer (ablation) | AudioMosaic linear probe · AS-20K | mAP: 30.2 -> 33.5 | Improved |
| time-frequency masking at inference instead of no masking when measuring effective rank (ablation) | encoders · effective rank analysis | effective rank | Improved |
| use time-frequency masking instead of unstructured masking for contrastive positive pairs (ablation) | AudioMosaic · AS-20K fine-tuning | mAP | Improved |
| add frequency masking on top of time masking at high mask ratios (ablation) | AudioMosaic · AS-20K fine-tuning | mAP | Improved |
| time-only masking vs time-frequency at low mask ratios (ablation) | AudioMosaic · AS-20K fine-tuning | mAP | No clear effect |
| increase pre-training batch size up to 6144 (ablation) | AudioMosaic · AS-20K fine-tuning | mAP | Improved |
| use rho_t 0.6 and rho_f 0.4 masking ratios (ablation) | AudioMosaic · AS-20K fine-tuning | mAP: 39.0 best cell in grid | Improved |
| use AASIST head instead of linear head on AudioMosaic (ablation) | AudioMosaic · EnvSDD TTA average | EER (%): 1.30 -> 1.41 | No clear effect |
| replace CAV-MAE encoder with AudioMosaic in LTU | LTU with LLaMA-7B · VGGSound | accuracy: 38.4 -> 54.6 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- SSLAM — SSLAM: Enhancing Self-Supervised Models with Audio Mixtures for Polyphonic Soundscapes (ICLR 2025) · better on 5 of 6 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- MATS: An Audio Language Model under Text-only Supervision (ICML 2025 · similarity 0.87)
- CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment (CVPR 2025 · similarity 0.86)
- CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processing (NeurIPS 2024 · similarity 0.86)
- Contrastive Learning from Synthetic Audio Doppelgängers (ICLR 2025 · similarity 0.86)
- Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs (NeurIPS 2024 · similarity 0.86)
- Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
