Research map / Vision-language and multimodal / Multimodal LLMs

AD-FM: Multimodal LLMs for Anomaly Detection via Multi-Stage Reasoning and Fine-Grained Reward Optimization

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Adapts Qwen2.5-VL-7B via LoRA and GRPO for industrial anomaly detection, using a three-stage think/rethink/answer reasoning format and a localization-aware reward (GIoU, box-count, normal-sample focus) to reduce zero-variance training samples.

Problem

General MLLMs perform poorly on industrial anomaly detection; existing GRPO-based approaches suffer from vanishing reward variance when responses are uniform and lack supervision over the reasoning process, allowing spuriously correct answers.

Approach

Combine a multi-stage (think, rethink, answer) reasoning format that yields diverse responses with a fine-grained reward that adds Hungarian-matched GIoU, box-count and normal-sample focus terms to the classification reward, so that correct-but-sloppy responses receive different rewards.

Key results

• Multi-domain average accuracy of 73.15% vs 51.04% for base Qwen2.5-VL-7B (+22.11%), above GPT-4.1 (71.60%) and OmniAD (68.80%).
• Cross-dataset (trained on MVTec) average 84.57% across VisA, MPDD, DTD, DAGM, above GPT-4.1 (81.98%) and best CLIP-based result with threshold tuning (83.67%).
• Zero-variance samples drop from 41.25% (binary reward) to 11.00% with the full fine-grained reward.

Surprising findings

• A random reward perturbation of similar variance yields only +0.40% over multi-stage binary reward, suggesting the spatial meaning of the reward matters beyond variance.

Limitations

• Baselines AnomalyR1 and OmniAD reported only average numbers from their papers; per-dataset comparison not possible.
• Lower accuracy on VisA cross-dataset vs thresholded CLIP baseline due to subtle defects.
• Bounding boxes are pseudo-generated from segmentation masks, so localization supervision is approximate.

Experimental setup

Datasets: MVTec VisA MVTec-LOCO GoodsAD

Benchmarks: MVTec VisA MVTec-LOCO GoodsAD MPDD DTD DAGM

Metrics: binary classification accuracy zero-variance sample percentage

Baselines: GPT-4o GPT-4o-mini GPT-4.1 LLaVA-NeXT InternVL2 InternVL3 Qwen2.5-VL AnomalyR1 OmniAD WinCLIP AnomalyCLIP

Builds on: GRPO DeepSeek-R1 AnomalyR1 OmniAD MMAD

Closest papers by meaning

• Judo: A Juxtaposed Domain-oriented Multimodal Reasoner for Industrial Anomaly QA (ICLR 2026 · similarity 0.88)
• MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection (ICLR 2025 · similarity 0.88)
• Towards Training-free Anomaly Detection with Vision and Language Foundation Models (CVPR 2025 · similarity 0.87)
• Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.85)
• LogicAD: Explainable Anomaly Detection via VLM-based Text Feature Extraction (AAAI 2025 · similarity 0.85)
• Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping (CVPR 2024 · similarity 0.85)