Research map / Label-efficient and robust learning / Self-supervised and contrastive
SSAST: Self-Supervised Audio Spectrogram Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pretrain AST with joint discriminative and generative masked-patch objectives | AST · AudioSet-20K | mAP: 14.8 -> 31.0 | Helped |
| use only the discriminative objective rather than the joint objective (ablation) | SSAST · AudioSet-20K | mAP: 31.0 -> 30.6 | Hurt |
| use only the generative objective rather than the joint objective (ablation) | SSAST · AudioSet-20K | mAP: 31.0 -> 16.1 | Hurt |
| pretrain with 100 masked patches instead of 400 (ablation) | SSAST · AudioSet-20K | mAP: 31.0 -> 28.7 | Hurt |
| pretrain with 250 masked patches instead of 400 (ablation) | SSAST · AudioSet-20K | mAP: 31.0 -> 30.4 | Hurt |
| pretrain on both AudioSet-2M and Librispeech rather than AudioSet-2M alone (ablation) | SSAST · AudioSet-20K | mAP: 29.0 -> 31.0 | Helped |
| pretrain on both AudioSet-2M and Librispeech rather than Librispeech alone (ablation) | SSAST · VoxCeleb 1 | accuracy: 60.8 -> 64.3 | Helped |
| use MSPM pretraining rather than training from scratch (ablation) | Tiny AST · AudioSet-20K | mAP: 15.1 -> 27.1 | Helped |
| use MSPM pretraining rather than training from scratch (ablation) | Small AST · AudioSet-20K | mAP: 16.5 -> 30.8 | Helped |
| use frame-like patches instead of square patches (ablation) | SSAST · AudioSet-20K | mAP: 31.0 -> 29.2 | Hurt |
| use frame-like patches instead of square patches (ablation) | SSAST · Speech Commands V1 | accuracy: 96.0 -> 96.7 | Helped |
| use MSPM pretraining rather than training from scratch | AST · ESC-50 | accuracy: 41.9 -> 88.8 | Helped |
Closest papers by meaning
- DTF-AT: Decoupled Time-Frequency Audio Transformer for Event Classification (AAAI 2024 · similarity 0.89)
- Masked Autoencoders that Listen (NeurIPS 2022 · similarity 0.87)
- MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training (ICLR 2024 · similarity 0.86)
- Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs (NeurIPS 2024 · similarity 0.85)
- SelfVC: Voice Conversion With Iterative Refinement using Self Transformations (ICML 2024 · similarity 0.84)
- SSLAM: Enhancing Self-Supervised Models with Audio Mixtures for Polyphonic Soundscapes (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.