Research map / Vision-language and multimodal / Visual question answering

WMarkGPT: Watermarked Image Understanding via Multimodal Large Language Models

ICML 2025 · Vision-language and multimodal › Visual question answering · approach: Multimodal LLM question answering

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes WMarkGPT, an MLLM (mPLUG-owl2-based) trained in three stages to describe watermark location/appearance/impact and predict watermark visibility from a watermarked image alone, with new WQA-Synthetic, WQA-Real and object-location VQA datasets.

Problem

Assessing watermark efficacy (especially visibility) in text-driven generative watermarking, where original images are unavailable; PSNR/SSIM/LPIPS need references and miss visibility and semantic corruption.

Approach

Train an MLLM with a three-stage curriculum (object-position pretraining on COCO, synthetic watermark VQA, real watermark VQA) so it describes watermarked images and derives a continuous visibility score from the softmax over the five visibility-level tokens, without reference images.

Key results

• On WQA-Real, WMarkGPT reaches BLEU-1 0.424, ROUGE-L 0.418, LLM-Score 71.95 and ACC 0.546, versus best baselines of about 0.279, 0.277, 17.15 and 0.110 for mPLUG-owl2 (reported as ~29%, 45%, 150%, 217% gains).
• Visibility score from token-probability weighting reaches SRCC 0.71 on WQA-Real versus 0.63 for the best score-based IQA baseline DBCNN (SRCC 0.63).
• Traditional metrics PSNR, SSIM and LPIPS are shown to be poorly aligned with human visibility ratings in the paper's correlation analysis.

Measured effects reported in the paper

Surprising findings

• Traditional metrics such as PSNR can assign favorable values to clearly visible watermarks, misaligned with human perception.
• The visibility score is trained only from coarse level descriptions, yet correlates well with human scores that were not used for supervision.

Limitations

• Evaluation on a relatively small real-watermark test set (about 0.5K images) and only five watermarking methods for WQA-Real.
• Relies on GPT-4 judge for ACC and LLM-Score and on GPT-4/GPT-4o-generated synthetic descriptions.

Experimental setup

Datasets: COCO LOGO-2K WQA-Synthetic WQA-Real object location-aware QA dataset

Benchmarks: WQA-Synthetic WQA-Real

Metrics: BLEU-1 ROUGE-L LLM-Score ACC SRCC PLCC KRCC

Baselines: Qwen2-VL-7B-Instruct Qwen-VL-Chat LLaVA-1.5-7B LLaVA-1.5-13B VILA-8B mPLUG-owl2-7B DBCNN HyperIQA TReS CLIP-IQA

Builds on: mPLUG-owl2 LLaVA-1.5 SAM COCO

Closest papers by meaning

• SleeperMark: Towards Robust Watermark against Fine-Tuning Text-to-image Diffusion Models (CVPR 2025 · similarity 0.88)
• Lightweight-Mark: Rethinking Deep Learning-Based Watermarking (ICML 2025 · similarity 0.88)
• Black-Box Detection of Language Model Watermarks (ICLR 2025 · similarity 0.88)
• Mask Image Watermarking (NeurIPS 2025 · similarity 0.87)
• Learning to Watermark: A Selective Watermarking Framework for Large Language Models via Multi-Objective Optimization (NeurIPS 2025 · similarity 0.87)
• On the Learnability of Watermarks for Language Models (ICLR 2024 · similarity 0.87)