Research map / Vision-language and multimodal / Cross-modal learning
Securing Multimodal AI through Internal Information Decomposition
Jehyeok Yeon, Hyeonjeong Ha, Qiusi Zhan, Heng Ji
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove Synergy feature from FlowVector (ablation) | FlowGuard · VLSafe | AUC: -0.125 on VLSafe, -0.110 on VLSU when removed | Improved |
| include Visual Uniqueness (Uv) feature (ablation) | FlowGuard · VizWiz (benign OOD) | AUC: prevents a -0.150 drop | Improved |
| use FlowVector relational features instead of raw final-layer embeddings with the same anomaly detector (ablation) | FlowGuard · MM-SafetyBench UMK, LLaVA-1.5-7B | ASR: 52.8 -> 8.7 | Improved |
| monitor more decoding steps (k=1 to k=10) (ablation) | FlowGuard · Detection AUC | AUC: 0.942 -> 0.948, latency 1.3s -> 12.8s | No clear effect |
| monitor k=3 decoding steps instead of 1 (ablation) | FlowGuard · Adversarial samples | evasion catch rate: 2.1% of evading samples caught at k=3 | Improved |
| use supervised MLP instead of Isolation Forest on FlowVectors (ablation) | FlowGuard · Unseen attacks | AUC: Isolation Forest AUC >= 0.88 on OOD | Hurt |
| add FlowGuard to undefended model against co-optimized GCG+PGD attack | LLaVA-1.5-7B · Co-optimized GCG and PGD attack | ASR: 86.4% -> 11.2% | Improved |
| FlowGuard instead of perplexity or confidence-based detection | Detector · Benign inputs | FPR: 2.4% vs 11.2% (perplexity), 14.5% (confidence) | Improved |
| use FlowGuard on text-only attacks versus visual or cross-modal attacks | FlowGuard · Text attacks | ASR: residual ASR about 13-14% text vs 6-9% visual/cross-modal | Mixed |
| generative verification defenses (CIDER, MirrorCheck) on low-quality images | CIDER and MirrorCheck · VizWiz-VQA | accuracy: 56.2 -> 35.2 (CIDER), 56.2 -> 38.1 (MirrorCheck) | Hurt |
Closest papers by meaning
- [Re] Boosting the Visual Interpretability of CLIP via Adversarial Fine-Tuning (NeurIPS 2026 · similarity 0.88)
- XFeat Revisited: Reproducibility and Evaluation of a Lightweight Image Matcher (NeurIPS 2026 · similarity 0.87)
- GuardAlign: Robust Safety Alignment in Multimodal Large Language Models (ICLR 2026 · similarity 0.87)
- Debate with Images: Detecting Deceptive Behaviors in Multimodal Large Language Models (ICML 2026 · similarity 0.87)
- CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolution (AAAI 2026 · similarity 0.87)
- Revisiting "Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing" (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
