Research map / Detection and segmentation / Open-set and few-shot detection
Forensics Adapter: Adapting CLIP for Generalizable Face Forgery Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Forensics Adapter, a lightweight tiny-ViT adapter running parallel to a frozen CLIP ViT-L/14 that learns blending-boundary traces with task-specific losses and feeds knowledge back into CLIP via attention bias, reaching 5.7M trainable parameters and about 7% average AUC gain for face forgery detection.
Problem
Generalizable face forgery detection: CLIP-based detectors use CLIP merely as a frozen feature extractor with generic adapters, lacking task-specific adaptation and interaction with CLIP's visual tokens.
Approach
A parallel tiny-ViT adapter trained with blending-boundary prediction and patch-wise/sample-wise contrastive losses, which interacts with frozen CLIP through fused shallow features and a learned attention bias on CLIP [CLS*] tokens, plus trainable 1x1 convs for CLIP-side contrastive learning.
Key results
- Frame-level cross-dataset AUC average 0.896 on five datasets trained on FF++ c23, about 7% above the previous best LSDA (0.826).
- Video-level AUC of 0.957 on CDF-v2, 0.872 on DFDC and 0.929 on DFDCP, surpassing prior methods such as LAA-NET and SBI.
- Uses 5.7M trainable parameters, while compared methods use 19M to 36M.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the Forensics Adapter instead of the strongest listed baseline | Face forgery detection trained on FF++ c23 · CDF-v1, CDF-v2, DFDC, DFDCP, DFD (frame-level) | AUC: 0.867 -> 0.914; 0.864 -> 0.900; 0.736 -> 0.843; 0.851 -> 0.890; 0.915 -> 0.933 | Helped |
| Use the Forensics Adapter instead of the strongest listed baseline | Face forgery detection · CDF-v2, DFDC, DFDCP (video-level) | AUC: 0.954 -> 0.957; 0.770 -> 0.872; 0.869 -> 0.929 | Helped |
| Remove masked blending boundary learning (L1) (ablation) | Forensics Adapter · CDF-v2 | AUC: 0.900 -> 0.899 | No clear effect |
| Remove patch-wise contrastive learning (L2) (ablation) | Forensics Adapter · CDF-v2 | AUC: 0.822 -> 0.900 | Hurt |
| Remove sample-wise contrastive learning (L3) (ablation) | Forensics Adapter · CDF-v2 | AUC: 0.899 -> 0.900 | No clear effect |
| Add task-specific learning on the adapter only (ablation) | Forensics Adapter · CDF-v2 | AUC: 0.789 -> 0.822 | Helped |
| Add task-specific learning on CLIP only (ablation) | Forensics Adapter · CDF-v2 | AUC: 0.789 -> 0.875 | Helped |
| Use task-specific learning on both adapter and CLIP (ablation) | Forensics Adapter · CDF-v2 | AUC: 0.875 -> 0.900 | Helped |
| Replace CLIP ViT-L/14 with CLIP ViT-B/16 (ablation) | Forensics Adapter · CDF-v2 | AUC: 0.900 -> 0.837 | Hurt |
| Replace CLIP ViT-L/14 with BLIP ViT-L/14 (ablation) | Forensics Adapter · CDF-v2 | AUC: 0.900 -> 0.607 | Hurt |
| Use LoRA instead of the task-specific adapter (ablation) | CLIP face forgery detection · CDF-v2 | AUC: 0.797 -> 0.900 | Hurt |
| Use subset fine-tuning instead of the task-specific adapter (ablation) | CLIP face forgery detection · CDF-v2 | AUC: 0.770 -> 0.900 | Hurt |
Surprising findings
- Adding a text modality with learnable CoOp-style prompts degraded average AUC from 0.896 to 0.744.
- A larger adapter (ViT-large) did not improve over the tiny ViT adapter under the same training configuration.
Limitations
- Designed for face forgery only; performance on fully synthesized GAN and diffusion faces is weaker (e.g., 0.638 AUC on PNDM).
- Relies on blending-boundary ground truth derived from manipulation masks.
- Trained only on FF++ c23 in the cross-dataset protocol.
Experimental setup
Datasets: FaceForensics++ c23
Benchmarks: CDF-v1 CDF-v2 DFDC DFDCP DFD StarGAN CramerGAN MMDGAN ADM PNDM
Metrics: AUC AP EER
Baselines: LSDA CFM ED SBI UCF Vanilla CLIP VLFFD FFAA CLIPping IID
Builds on: CLIP CoOp LoRA
Closest papers by meaning
- Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector (CVPR 2025 · similarity 0.89)
- Identity-Aware Vision-Language Model for Explainable Face Forgery Detection (AAAI 2026 · similarity 0.88)
- Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection (CVPR 2024 · similarity 0.88)
- DevFD : Developmental Face Forgery Detection by Learning Shared and Orthogonal LoRA Subspaces (NeurIPS 2025 · similarity 0.87)
- Stacking Brick by Brick: Aligned Feature Isolation for Incremental Face Forgery Detection (CVPR 2025 · similarity 0.87)
- A Hitchhiker's Guide to Fine-Grained Face Forgery Detection Using Common Sense Reasoning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.