Research map / Continual learning / Online and replay-based

What to Remember: Self-Adaptive Continual Learning for Audio Deepfake Detection

AAAI 2024 · Continual learning › Online and replay-based · approach: Dynamic expansion and isolation · 30 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Radian Weight Modification (RWM), a continual learning method that splits classes by in-class cosine compactness and learns self-attention-based gradient modification angles, evaluated on audio deepfake detection (ASVspoof2019LA, ASVspoof2015, In-the-Wild) and CLEAR image recognition.

Problem

Audio deepfake detectors degrade when new attack types appear, and existing continual learning methods either need stored old data or sacrifice learning of new tasks.

Approach

Partition classes into compact (e.g., genuine audio) and dispersed (fake audio) groups by average in-class cosine distance, and use a self-attention-derived learned rotation angle to modify gradients, orthogonal to the old-task subspace for dispersed classes and nearly unmodified for compact classes, building on OWM.

Key results

• On the S→T1 two-dataset scenario, RWM reached 0.438% EER on S and 0.212% on T1, versus 7.324% and 0.510% for Finetune.
• On CLEAR after all 10 experiences, RWM (r_s=4) reached 95.25% on Exp10 and 93.13% on Exp6, second only to Replay-All (96.16% on Exp10) and above OWM and EWC; the paper reports r_s=4 as the best setting.
• On the S→T2 scenario, RWM reduced EER on T2 to 2.247% versus 4.978% for Finetune and 6.275% for DFWF, though S EER was 3.665%.

Measured effects reported in the paper

Surprising findings

• RWM's accuracy on CLEAR was lower than most other continual learning methods before experience 8, attributed to the model having to learn the modification direction itself.
• Reducing training data for the new task from 100 to 10 samples degraded RWM's ability to mitigate forgetting.

Limitations

• Lower accuracy than most baselines on early CLEAR experiences because the model must learn the gradient modification direction itself.
• Performance depends on the regrouping hyperparameter r_s, which is not automatically determined.
• Limited training data for new tasks (10 samples) weakens mitigation of forgetting.

Experimental setup

Datasets: ASVspoof2019LA ASVspoof2015 In-the-Wild VCTK CLEAR

Benchmarks: ASVspoof2019LA ASVspoof2015 In-the-Wild CLEAR

Metrics: EER accuracy

Baselines: Finetune EWC OWM LwF DFWF Replay-All GDF CWR GEM AGEM SI BRF

Builds on: Orthogonal Weight Modification (OWM) Elastic Weight Consolidation (EWC) Learning without Forgetting (LwF) Detecting Fake Without Forgetting (DFWF)

Closest papers by meaning

• Region-Based Optimization in Continual Learning for Audio Deepfake Detection (AAAI 2025 · similarity 0.91)
• Adaptive Compression and Targeted Perturbation: A Unified Framework for Generalized Audio Deepfake Detection (NeurIPS 2026 · similarity 0.86)
• Circumventing Shortcuts in Audio-visual Deepfake Detection Datasets with Unsupervised Learning (CVPR 2025 · similarity 0.86)
• I Can Hear You: Selective Robust Training for Deepfake Audio Detection (ICLR 2025 · similarity 0.85)
• AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection (CVPR 2024 · similarity 0.84)
• Multi-modal Deepfake Detection via Multi-task Audio-Visual Prompt Learning (AAAI 2025 · similarity 0.84)