Research map / Vision-language and multimodal / Multimodal LLMs

TextShield-R1: Reinforced Reasoning for Tampered Text Detection

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TextShield-R1, an RL-trained MLLM (Qwen2.5-VL-7B) for tampered text detection and reasoning with forensic continual pre-training, GRPO with five rewards, and OCR rectification at inference, plus the TFR benchmark of ~45k real and tampered text images.

Problem

Multimodal LLMs struggle to detect micro-level tampering artifacts in text images, localize tampered text regions accurately, and depend on costly forgery-interpretation annotations.

Approach

Pre-trains an MLLM on natural-image forgery data plus OCR grounding as an easy-to-hard curriculum, fine-tunes with GRPO using five task rewards (classification, method, localization IoU, OCR similarity, format) to reduce annotation needs, and refines predicted text boxes by matching MLLM-recognized text to an external OCR engine's boxes.

Key results

• On the TFR test set, TextShield-R1 reaches 88.1 classification accuracy, 47.6 OCR accuracy, 57.8 localization IoU and 58.8 reasoning score, versus 79.1/24.3/18.2/42.9 for the Qwen2.5-VL-7B fine-tuned baseline.
• Ablation: removing OCR Rectification drops test-set localization from 57.8 to 42.7 while keeping other metrics nearly unchanged.
• Removing Forensic Continual Pre-training drops the full model's test-set localization from 57.8 to 12.7 and falls below the fine-tuned baseline on several metrics.

Surprising findings

• Removing Forensic Continual Pre-training causes performance below the plain Qwen2.5-VL-7B baseline, indicating that GRPO fine-tuning fails to converge without this stage.
• The full model with only about 25% fully annotated reasoning data performs comparably to the variant trained with full annotations.

Limitations

• OCR Rectification depends on an external OCR engine and a fixed Levenshtein threshold of 0.2.
• Reasoning annotations for the TFR benchmark were generated by GPT-4o and then manually refined, so annotation quality depends on that pipeline.

Experimental setup

Datasets: CASIAv1v2 IMD20 NIST16 MIML Community Forensic COCO LAION TFR

Benchmarks: TFR test set TFR CIS set TFR CTM set TFR CL set

Metrics: classification accuracy OCR accuracy localization IoU reasoning score (cosine similarity, ROUGE-L, BLEU)

Baselines: GPT-4o MiniCPM-V-2.6 InternVL3-2B InternVL3-8B Qwen2.5-VL-3B Qwen2.5-VL-7B FakeShield SIDA

Builds on: GRPO Qwen2.5-VL FakeShield SIDA LoRA

Closest papers by meaning

• Revisiting Tampered Scene Text Detection in the Era of Generative AI (AAAI 2025 · similarity 0.90)
• Toward Real-world Text Image Forgery Localization: Structured and Interpretable Data Synthesis (NeurIPS 2025 · similarity 0.89)
• FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models (ICLR 2025 · similarity 0.87)
• Forensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Models (CVPR 2025 · similarity 0.87)
• TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection (NeurIPS 2026 · similarity 0.86)
• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL (ICML 2026 · similarity 0.86)