Research map / Detection and segmentation / Open-set and few-shot detection
SONICS: Synthetic Or Not - Identifying Counterfeit Songs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SONICS, a large dataset of end-to-end AI-generated songs (Suno/Udio) vs. real YouTube songs, and SpecTTTra, a spectro-temporal token transformer for efficient long-audio synthetic song detection.
Problem
Detecting end-to-end synthetic songs (where vocals, instrumentals, lyrics and style are all AI-generated) is not addressed by existing singing voice deepfake datasets, which are short, lack music-lyrics diversity, and keep real background music.
Approach
A large dataset of full-length end-to-end AI songs from Suno and Udio with varied lyrics/style combinations, plus SpecTTTra, which tokenizes a mel-spectrogram separately along time and frequency so that global attention over long songs stays cheap compared with ViT patch tokens.
Key results
- On 120-second songs, SpecTTTra-alpha reaches F1 0.96-0.97, outperforming ViT (0.82) and ConvNeXt (0.95) on the overall F1 metric.
- SpecTTTra-alpha is 38% faster than ViT and uses 26% less memory on 120-second audio, per the efficiency benchmark.
- Human listeners achieve overall F1 of 0.71, below all AI models (ConvNeXt 0.92 best on the 25-second human-benchmark clips).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 120-sec rather than 5-sec audio | ConvNeXt · SONICS test | F1: 0.90 -> 0.96 | Helped |
| Use 120-sec rather than 5-sec audio | ViT · SONICS test | F1: 0.79 -> 0.89 | Helped |
| Use 120-sec rather than 5-sec audio | EfficientViT · SONICS test | F1: 0.87 -> 0.95 | Helped |
| Use 120-sec rather than 5-sec audio | SpecTTTra-α · SONICS test | F1: 0.80 -> 0.97 | Helped |
| Add spectral tokens to temporal tokens (f = 1) (ablation) | SpecTTTra-α · SONICS test, 5 sec | F1: 0.76 -> 0.80 | Helped |
| Add spectral tokens to temporal tokens (f = 1) (ablation) | SpecTTTra-α · SONICS test, 120 sec | F1: 0.91 -> 0.97 | Helped |
| Add temporal tokens to spectral tokens (t = 3) (ablation) | SpecTTTra-α · SONICS test, 5 sec | F1: 0.75 -> 0.80 | Helped |
| Add temporal tokens to spectral tokens (t = 3) (ablation) | SpecTTTra-α · SONICS test, 120 sec | F1: 0.92 -> 0.97 | Helped |
| Reduce spectral clip size from 5 to 3 (ablation) | SpecTTTra-α · SONICS test, 5 sec | F1: 0.78 -> 0.79 | Helped |
| Reduce spectral clip size from 3 to 1 (ablation) | SpecTTTra-α · SONICS test, 5 sec | F1: 0.79 -> 0.80 | Helped |
| Reduce spectral clip size from 5 to 3 (ablation) | SpecTTTra-α · SONICS test, 120 sec | F1: 0.94 -> 0.96 | Helped |
| Reduce spectral clip size from 3 to 1 (ablation) | SpecTTTra-α · SONICS test, 120 sec | F1: 0.96 -> 0.97 | Helped |
Surprising findings
- Models trained on SingFake drop sharply on SONICS (e.g., ConvNeXt F1 0.86 to 0.33), showing SVDD detectors do not transfer to end-to-end songs.
- ConvNeXt outperforms the proposed SpecTTTra on short 5-second clips, while SpecTTTra only wins on long audio.
Limitations
- Real songs retrieved from YouTube by title/artist search may be mismatched in about 0.5% of cases.
- Only English songs and only smaller model variants were evaluated; models trained from scratch rather than pretrained.
- Generalization to unseen generators (SkyMusic, SeedMusic) is weak, with several models falling to F1 around 0.2-0.6.
Experimental setup
Datasets: SONICS Genius Lyrics Dataset SingFake SkyMusic (external test) SeedMusic (external test)
Benchmarks: SONICS
Metrics: F1 sensitivity specificity EER speed (A/S) FLOPs memory parameter count
Baselines: ConvNeXt ViT EfficientViT AASIST Wav2Vec2 + AASIST ResNet + LFCC ResNet + Spectrogram
Builds on: ViT ConvNeXt EfficientViT SingFake
Closest papers by meaning
- Aria-MIDI: A Dataset of Piano MIDI Files for Symbolic Music Modeling (ICLR 2025 · similarity 0.82)
- Detecting Music Performance Errors with Transformers (AAAI 2025 · similarity 0.82)
- Contrastive Learning from Synthetic Audio Doppelgängers (ICLR 2025 · similarity 0.82)
- SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation (ICML 2025 · similarity 0.82)
- SonicSim: A customizable simulation platform for speech processing in moving sound source scenarios (ICLR 2025 · similarity 0.82)
- Transcoda: End-to-End Zero-Shot Optical Music Recognition via Data-Centric Synthetic Training (NeurIPS 2026 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.