Research map / Vision-language and multimodal / Multimodal LLMs

OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes OmniVL-Guard, a Qwen3VL-8B LoRA model trained with self-evolving CoT data (FSFR) and ARSPO, a reward-scaling and dynamic task-weighting RL method, for unified text/image/video forgery detection and grounding.

Problem

Unified forgery detection and localization across interleaved text, image, and video misinformation, where multi-task RL is dominated by easier classification and under-optimizes fine-grained grounding ("difficulty bias").

Approach

ARSPO combines task-specific exponential reward mapping for localization tasks (steeper slope amplifies high-quality responses) with a dynamic coefficient scheme that periodically reweights task losses based on gains and trends, built on SAPO; training data comes from a self-evolving CoT pipeline with multi-agent hard-CoT synthesis.

Key results

• In-domain, OmniVL-Guard reaches 54.26 IoU on image localization, 63.78 F1 on text localization, and 59.22 tIoU on video localization, versus 51.08, 44.67, and 33.08 for the SFT baseline.
• Ablation: adding SAPO, TBRMF, and DCA raises the average gain over SFT from +24.33 to +28.33 points.
• Zero-shot OOD: 93.69 on ISOT text, 63.64 on CASIA2.0, 79.38 on MMFakeBench, and 63.55 on FakeSV.

Surprising findings

• Plain GRPO raises binary classification accuracy by 36 points over SFT but leaves image localization essentially unchanged (-0.1), showing the imbalance the paper targets.
• Direct injection of ground-truth labels when generating CoT gives weaker downstream results than self-evolving CoT, attributed to hindsight bias in the reasoning.

Limitations

• Dataset access is restricted to academic use via an application-and-review process, so the dataset is not fully open.
• The theoretical gradient analysis uses a direct-path approximation that ignores cross-sample normalization terms.
• Dynamic coefficient hyperparameters are hand-set heuristics (thresholds, decay and boost factors).

Experimental setup

Datasets: FakeNewsCorpus MCFEND FakeClue LOKI ForgeryNet GenVideo DVF SAMM MDSM DGM^4 NewsCLIPpings

Benchmarks: FSFR in-domain test set ISOT CASIA2.0 MMFakeBench FakeSV

Metrics: accuracy IoU F1 tIoU

Baselines: GRPO SAPO Qwen3VL-235B Llama3.2-90B HAMMER FKA-Owl AMD D^3 Fake-VLM FakeSV-VLM Video-R1 Bert

Builds on: GRPO SAPO DeepSeek-R1 Qwen3-VL

Closest papers by meaning

• Identity-Aware Vision-Language Model for Explainable Face Forgery Detection (AAAI 2026 · similarity 0.88)
• Forensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Models (CVPR 2025 · similarity 0.88)
• A Hitchhiker's Guide to Fine-Grained Face Forgery Detection Using Common Sense Reasoning (NeurIPS 2024 · similarity 0.88)
• Unlocking the Capabilities of Large Vision-Language Models for Generalizable and Explainable Deepfake Detection (ICML 2025 · similarity 0.87)
• SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints (AAAI 2025 · similarity 0.86)
• Towards General Visual-Linguistic Face Forgery Detection (CVPR 2025 · similarity 0.86)