Research map / Vision-language and multimodal / Multimodal LLMs

ICM-Assistant: Instruction-tuning Multimodal Large Language Models for Rule-based Explainable Image Content Moderation

AAAI 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds a rule-based image content moderation dataset (ICM-Instruct) via decomposed attribute products and multi-stage MLLM/LLM prompting, then instruction-tunes several open MLLMs into ICM-Assistant models for explainable moderation.

Problem

Existing MLLMs are not aligned with specific moderation rules, producing classification decisions and explanations inconsistent with human moderators; traditional ICM classifiers lack explainability and flexibility across rules.

Approach

Decompose moderation rules into attribute products to create short human-written explicit descriptions, then use multi-stage prompts with pretrained MLLMs and LLMs to enrich them into moderation explanations and Q-A pairs, and instruction-tune MLLMs on the resulting dataset (CSFT or MSFT strategies).

Key results

• ICM-LLaVA-v1.5-R1 reaches 0.987 ICM accuracy on ICM-Val-R1 vs 0.794 for LLaVA-v1.5 (+24.3%).
• Average MEQ improvement of 26.6% over pre-trained MLLMs; ICM-LLaVA-v1.5 MEQ 0.844 vs 0.696 for LLaVA-v1.5 on ICM-Val-R1.
• On the UGC test set, ICM-LLaVA-v1.6 achieves 0.972 accuracy vs 0.929 for LLaVA-v1.6.

Measured effects reported in the paper

Surprising findings

• Multiple-choice Q-A questions boost ICM accuracy and MQA accuracy but restrict answer richness, lowering explanation quality.
• Traditional classifiers and open-source ICM models trained on limited terms like pornography generalize poorly to related terms such as sexy.

Limitations

• Main experiments limited to a single moderation term ('sexy') with two rules; balance across terms left to future work.
• Explanation quality judged by LLMs compared against 379 human-labeled explanations, with unspecified judge model.
• Some dataset components (test/annotations) rely on human moderators, limiting scalability.

Experimental setup

Datasets: ICM-Instruct NudeNet test set LLaVA-Bench AIIA-PID4 Adult content LSPD MemeGuard

Benchmarks: ICM-Val-R1 ICM-Val-R2 ICM-Test (UGC, AIGC) NudeNet test set LLaVA-Bench horrifying and gambling test sets

Metrics: ICM accuracy moderation explanation quality (MEQ) moderation question answering accuracy (MQA acc.) L-B score

Baselines: LLaVA-v1.5 LLaVA-v1.6 Qwen-VL mPLUG-Owl2 BLIP2 Kosmos-2 SD Safety Checker NSFW model NSFW detection ICM-ResNet-152 ICM-DenseNet-201 ICM-CLIP-L/14-336

Builds on: LLaVA Chain-of-Thought prompting ShareGPT4V CogVLM2 Qwen2

Closest papers by meaning

• MLLM-as-a-Judge for Image Safety without Human Labeling (CVPR 2025 · similarity 0.87)
• CoMM: A Coherent Interleaved Image-Text Dataset for Multimodal Understanding and Generation (CVPR 2025 · similarity 0.87)
• A High Quality Dataset and Reliable Evaluation for Interleaved Image-Text Generation (ICLR 2026 · similarity 0.87)
• AIM: Let Any Multimodal Large Language Models Embrace Efficient In-Context Learning (AAAI 2025 · similarity 0.86)
• OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation (CVPR 2025 · similarity 0.86)
• MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning (ICLR 2025 · similarity 0.86)