AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

MUTE: Multi-Level Alignment Uncoupling Against Talking-Head Exploitation for Voice Protection

Donghyun Kim, Jin Hong, Seungmin Kim, Dain Kim, Junseok Kwon, Daeseon Choi

NeurIPS 2026 · Vision-language and multimodal › Cross-modal learning · approach: Audio-visual speech separation · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MUTE, an audio protection framework that perturbs speech to disrupt talking-head generation models' audio-visual alignment and prevent misuse in deepfakes.

Problem

Talking-head generation models synthesize realistic facial animation from audio and are vulnerable to misuse in multimodal deepfakes; effective protection of the audio input remains underexplored.

Approach

Uses multi-level alignment uncoupling: an audio protection perturbation is designed to break the audio-visual alignment that talking-head models rely on, so generated faces no longer follow the protected audio.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use MUTE instead of strongest audio baseline CAFaudio protection · TalkingHead-1KHSyncNet Sync: 2.173 -> 1.332Improved
use MUTE instead of CAFaudio protection · black-box SadTalkerSyncNet Sync: 5.425 -> 4.703Improved
add frame-wise embedding cosine loss instead of global (ablation)MUTE representation-level loss · TalkingHead-1KH (Hallo)SyncNet Sync: 2.484 -> 1.332Improved
add STFT loss (lambda=5) to MUTE (ablation)MUTE · clean and down/upsampled audioSyncNet Sync: 1.492 -> 1.332 (clean); 3.178 -> 2.914 (Down/Up)Improved
use second-order term in sensitivity score instead of first-order only (ablation)target attention · TalkingHead-1KH (Hallo)SyncNet Sync: 1.383 -> 1.332Improved
use two-phase alignment-then-representation optimization (rho=0.1) vs single objective (ablation)MUTE · Hallo, Audio2Head, SadTalkerSyncNet Sync: rho=1.0: 3.920/3.958/5.048; rho=0.0: 1.427/3.904/5.063; rho=0.1: 1.332/3.627/4.703Improved
increase perturbation budget epsilon from 0.004 to 0.010 (ablation)MUTE · TalkingHead-1KHSyncNet Sync and audio quality: Sync 1.332 -> 1.027; WER 11.68 -> 13.30; PESQ 2.522 -> 1.945Mixed
add speaker-level loss lambda_spk=0.01 (ablation)MUTE · TTS resynthesis (CosyVoice, XTTS-v2, Chatterbox)defense success rate: DSR 14.67 -> 67.73 (CosyVoice); Sync 1.332 -> 1.496Improved
combine MUTE with image-based protection SilencerSilencer · image+audio protectionSyncNet Sync: 2.170 -> 0.760Improved
apply adaptive purification attackMUTE · adaptive attackSyncNet Sync: 1.332 -> 4.269Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy