Research map / Vision-language and multimodal / Cross-modal learning

Suppress and Rebalance: Towards Generalized Multi-Modal Face Anti-Spoofing

CVPR 2024 · Vision-language and multimodal › Cross-modal learning · approach: Multi-modal object re-identification · 36 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MMDG, a ViT-based multi-modal face anti-spoofing framework with uncertainty-guided cross-adapters (U-Adapter) and gradient-modulation rebalancing (ReGrad), plus the first multi-source domain generalization benchmark for multi-modal FAS.

Problem

Multi-modal face anti-spoofing generalizes poorly to unseen attacks and deployment conditions because depth/infrared modalities cause unreliable cross-modal fusion under domain shift and training suffers from modality imbalance.

Approach

Use Monte Carlo dropout uncertainty to down-weight unreliable tokens in cross-modal attention fusion (U-Adapter), and adaptively modulate each modality's gradients based on gradient conflict and convergence speed, with a single-side prototypical loss to measure modality convergence (ReGrad).

Key results

• On Protocol 1 leave-one-out sub-protocols, MMDG reaches average HTER 19.25% and AUC 87.96%, versus 34.22% HTER for SSDG and 25.45% HTER for VP-FAS (best prior).
• On Protocol 2 with missing modalities, MMDG achieves average HTER 24.51% and AUC 82.69%, outperforming VP-FAS (29.74% HTER).
• Ablation on Protocol 1: ViT+U-Adapter with ReGrad reaches HTER 19.48% and AUC 87.48%, compared with 27.38% HTER without ReGrad.

Measured effects reported in the paper

Surprising findings

• Multi-modal extensions of SSDG performed worse than its unimodal RGB version (AUC drop from 83.69 to 82.03), showing more modalities do not automatically improve generalization.
• Several methods (SSAN, IADG, ViT, ViT+AMA) performed better when both depth and infrared were missing than when only one was missing, suggesting unreliable modality information misleads detection.

Limitations

• Modality balance is not fully achieved; the authors note further enhancement of modality balance is needed.
• Uncertainty estimation via Monte Carlo dropout may not be the most efficient; the authors suggest more efficient uncertainty techniques.
• Performance on the benchmark remains far from perfect (average HTER around 19%), leaving room for improvement.

Experimental setup

Datasets: CASIA-CeFA PADISI-Face CASIA-SURF WMCA

Benchmarks: CASIA-CeFA PADISI-Face CASIA-SURF WMCA

Metrics: HTER AUC

Baselines: SSDG SSAN IADG ViTAF MM-CDCN CMFL ViT + AMA VP-FAS ViT PMR OGM-GE

Builds on: SSDG PMR OGM-GE Vision Transformer central difference convolution

Closest papers by meaning

• FaceShield: Explainable Face Anti-Spoofing with Multimodal Large Language Models (AAAI 2026 · similarity 0.84)
• mmFAS: Multimodal Face Anti-Spoofing Using Multi-Level Alignment and Switch-Attention Fusion (AAAI 2025 · similarity 0.83)
• Gradient Alignment for Cross-Domain Face Anti-Spoofing (CVPR 2024 · similarity 0.83)
• From Intuition to Investigation: A Tool-Augmented Reasoning MLLM Framework for Generalizable Face Anti-Spoofing (CVPR 2026 · similarity 0.82)
• Interpretable Face Anti-Spoofing: Enhancing Generalization with Multimodal Large Language Models (AAAI 2025 · similarity 0.82)
• Rethinking Generalizable Face Anti-spoofing via Hierarchical Prototype-guided Distribution Refinement in Hyperbolic Space (CVPR 2024 · similarity 0.82)