Research map / Vision-language and multimodal / Cross-modal learning
mmFAS: Multimodal Face Anti-Spoofing Using Multi-Level Alignment and Switch-Attention Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes mmFAS, a multimodal (RGB, depth, IR) face anti-spoofing framework with multi-level instance/class alignment and a parameter-free switch-attention fusion, trained on CeFa and tested cross-dataset on MmFA, WMCA, and HQ-WMCA.
Problem
Face anti-spoofing with multiple visual modalities generalizes poorly because naive fusion ignores latent feature gaps between RGB, depth, and infrared representations.
Approach
Explicitly align modality features before fusion using instance-level contrastive learning plus class-level pair matching with hard negative/positive mining, with RGB as a bridge, then fuse them via a parameter-free switch-attention that cyclically permutes queries/keys/values across the three modality branches.
Key results
- mmFAS achieves the best weighted average results across four datasets, with weighted HTER 0.0183%, AUC 99.93%, and Acc 79.24% versus the next best ViT-S/16 (0.0546 / 99.27 / 78.97).
- On the cross-dataset MmFA test, mmFAS reports 9.24% HTER and 97.80% AUC, and on HQ-WMCA it reports 18.84% HTER and 85.24% AUC.
- Ablation shows the full trimodal model with alignment and switch-attention reaches 67.14% MmFA accuracy and 79.24% weighted accuracy, and the RDI switch order performs best among six orders.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use mmFAS instead of the strongest listed baseline | FAS models · MmFA | HTER: 10.17 -> 9.24 | Helped |
| use mmFAS instead of the strongest listed baseline | FAS models · MmFA | AUC: 96.09 -> 97.80 | Helped |
| use mmFAS instead of the strongest listed baseline | FAS models · MmFA | Accuracy: 69.65 -> 67.14 | Hurt |
| use mmFAS instead of the strongest listed baseline | FAS models · WMCA | AUC: 59.77 -> 64.79 | Helped |
| use mmFAS instead of the strongest listed baseline | FAS models · HQ-WMCA | HTER: 31.04 -> 36.92 | Hurt |
| use all three modalities instead of a single modality (ablation) | mmFAS · MmFA | Accuracy: 52.31 -> 55.18 | Helped |
| use all three modalities instead of RGB and depth (ablation) | mmFAS · MmFA | Accuracy: 49.09 -> 55.18 | Helped |
| remove one or more modalities from the three-modality configuration (ablation) | mmFAS · MmFA | Weighted Accuracy: 74.10; 70.72; 68.68; 71.01; 71.51; 69.34; 60.41 | Mixed |
| use instance-level contrastive learning and class-level matching for alignment (ablation) | mmFAS · cross-dataset testing | Weighted Accuracy: 65.05; 56.36; 56.67; 65.35; 61.09 | Mixed |
| use switch-attention rather than merge-attention or cross-attention (ablation) | mmFAS · MmFA | Weighted Accuracy: 72.94; 72.98; 79.24 | Helped |
| change the switch-mode sequence from RDI to another order (ablation) | mmFAS · cross-dataset testing | AUC: 85.24 -> 82.28; 82.18; 84.74; 82.76; 82.32 | Hurt |
| vary α or β away from 1 (ablation) | mmFAS · cross-dataset testing | AUC | Hurt |
Surprising findings
- Multimodal models without alignment can generalize worse than unimodal models in cross-dataset tests.
- Some FAS-specific baselines (DeepPixBiS, ADeepPixBiS) perform worse than random on MmFA.
Limitations
- Cross-dataset generalization remains limited; several baselines drop near random performance on some datasets.
- HQ-WMCA depth modality is absent and synthesized with PRNet, and flat attacks have depth removed manually, so evaluation relies on constructed inputs.
Experimental setup
Datasets: CeFa MmFA WMCA HQ-WMCA
Benchmarks: CeFa MmFA WMCA HQ-WMCA
Metrics: HTER AUC Accuracy weighted accuracy
Baselines: DeepPixBiS ADeepPixBiS ResNet50 MLP-Mixer ConvMLP PipeNet FeatherNet FlexModal-FAS FaceBagNet ViT-S/16
Builds on: Vision Transformer merge-attention cross-attention DeepPixBiS FeatherNet
Closest papers by meaning
- FaceShield: Explainable Face Anti-Spoofing with Multimodal Large Language Models (AAAI 2026 · similarity 0.90)
- Interpretable Face Anti-Spoofing: Enhancing Generalization with Multimodal Large Language Models (AAAI 2025 · similarity 0.89)
- PA-FAS: Towards Interpretable and Generalizable Multimodal Face Anti-Spoofing via Path-Augmented Reinforcement Learning (AAAI 2026 · similarity 0.89)
- SLIP: Spoof-Aware One-Class Face Anti-Spoofing with Language Image Pretraining (AAAI 2025 · similarity 0.88)
- From Intuition to Investigation: A Tool-Augmented Reasoning MLLM Framework for Generalizable Face Anti-Spoofing (CVPR 2026 · similarity 0.88)
- Harnessing Chain-of-Thought Reasoning in Multimodal Large Language Models for Face Anti-Spoofing (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.