Research map / Adversarial robustness and security / Attacks on text and graphs

Silence is Golden: Leveraging Adversarial Examples to Nullify Audio Control in LDM-based Talking-Head Generation

CVPR 2025 · Adversarial robustness and security › Attacks on text and graphs · approach: Diffusion-based adversarial purification · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Silencer, a two-stage adversarial perturbation method that makes portraits "silent" under LDM-based audio-driven talking-head generation and adds anti-purification via DDIM-inverted latent optimization.

Problem

Proactive defenses against LDM-based mimicry do not stop audio-driven talking-head animation of reference portraits, and diffusion-based purification removes their perturbations.

Approach

A nullifying loss treats the reference portrait as the target so the talking-head denoiser ignores audio (no ground-truth video needed), followed by optimizing the DDIM-inverted latent of the portrait under a constraint from stage-I adversarial examples, with a facial mask, to resist purification.

Key results

• On CelebA-HQ, Silencer Stage I reduces SyncNet confidence to 4.0644 versus 6.4003 for SDTS(-) and 5.6150 for AdvDM(+) (higher Sync = more synchronization).
• Silencer Stage II lowers I-PSNR to 27.23 on CelebA-HQ under protection and gives the lowest I-PSNR under purification methods such as JPEG, DiffPure, and GrIDPure.
• Timestep range [200, 300] for Stage I gives sync confidence 4.2556 with I-PSNR 32.34 on CelebA-HQ subset.

Measured effects reported in the paper

Surprising findings

• Stage II optimization in latent space causes large visual distortion in the facial region, requiring a face mask that trades some purification robustness for face preservation.
• Perturbations from Stage II are not fully robust: purification alters their structure, so the portrait is not fully kept silent.

Limitations

• Purified perturbations are partly disrupted, so the portrait is not fully silent after purification.
• Stage II reduces image quality (lower I-PSNR and SSIM) and FID is not consistently best; the method only partly preserves video quality.
• Evaluation uses a single driving audio for CelebA-HQ and only 50 images/clips per benchmark.

Experimental setup

Datasets: CelebA-HQ TalkingHead-1KH

Benchmarks: CelebA-HQ TalkingHead-1KH

Metrics: V-PSNR SSIM FID SyncNet confidence (Sync) M-LMD I-PSNR I-SSIM

Baselines: AdvDM(+) AdvDM(-) PhotoGuard Mist SDS(+) SDS(-) SDTS(-) ACA

Builds on: AdvDM PhotoGuard Diff-Protect (SDS) ACA DiffPure

Closest papers by meaning

• DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing (ICLR 2025 · similarity 0.86)
• De-AntiFake: Rethinking the Protective Perturbations Against Voice Cloning Attacks (ICML 2025 · similarity 0.86)
• Unmasking Puppeteers: Leveraging Biometric Leakage to Expose Impersonation in AI-Based Videoconferencing (NeurIPS 2025 · similarity 0.86)
• Enhancing Facial Privacy Protection via Weakening Diffusion Purification (CVPR 2025 · similarity 0.85)
• Anti-Diffusion: Preventing Abuse of Modifications of Diffusion-Based Models (AAAI 2025 · similarity 0.85)
• Adversarial Robust Safeguard for Evading Deep Facial Manipulation (AAAI 2024 · similarity 0.85)