Research map / Vision-language and multimodal / Multimodal LLMs

Common Inpainted Objects In-N-Out of Context

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces COinCO, a 97,722-image dataset built by diffusion-based inpainting of COCO objects with LVLM-labeled in- and out-of-context annotations, and demonstrates context classification distillation, objects-from-context prediction, and context-enhanced fake localization.

Problem

Out-of-context examples are scarce in existing vision datasets, which limits learning of contextual inconsistency and hinders context-aware reasoning and image forensics.

Approach

Replace exactly one COCO object per image via Stable Diffusion inpainting with random category prompts, verify with YOLOv8x, and label context via unanimous consensus of three LVLMs on location, size and co-occurrence criteria; then use the labels for distillation, a new objects-from-context task, and a score-boosting scheme for existing fake localizers.

Key results

• Qwen2.5-VL-72B-based consensus agreed with humans on 95.10% of 1,000 sampled images; distilled 3B students raised average context accuracy on COinCO test from 66.2% to 73.7%.
• Objects-from-Context model reaches 16.32% instance top-1 and 35.10% clique top-1, versus 1.54% and 9.37% for co-occurrence baseline.
• CAT-Net achieves the best fake localization among evaluated detectors (AUC 97.4, AP 90.3 on COinCO).

Surprising findings

• Stronger inpainting models (Flux, Qwen Image) sacrificed out-of-context generation ability, so the authors used Stable Diffusion instead.
• Distilled co-occurrence model slightly decreased accuracy on original COCO images after fine-tuning (89.0% to 87.0%).

Limitations

• Out-of-context labels are restricted to COCO's 80 predefined categories.
• Inpainting verification with YOLOv8x has only 78.75% F1 and some false positives from low-quality inpaintings.
• Context labels are limited to unanimous-agreement images (73,929 of 97,722).

Experimental setup

Datasets: COCO 2017 COinCO

Benchmarks: COinCO COCO 2017 original images

Metrics: accuracy F1 AUC AP top-k accuracy

Baselines: Random Co-occurrence ManTraNet TruFor PSCC-Net CAT-Net

Builds on: COCO Stable Diffusion inpainting knowledge distillation

Closest papers by meaning

• ContextShift: A Controlled Benchmark for Context Dependence in Object Detection (NeurIPS 2026 · similarity 0.85)
• Paint by Inpaint: Learning to Add Image Objects by Removing Them First (CVPR 2025 · similarity 0.84)
• Brush2Prompt: Contextual Prompt Generator for Object Inpainting (CVPR 2024 · similarity 0.84)
• The Devil is in the Fine-Grained Details: Evaluating Open-Vocabulary Object Detectors for Fine-Grained Understanding (CVPR 2024 · similarity 0.84)
• The Devil is in the Object Boundary: Towards Annotation-free Instance Segmentation using Foundation Models (ICLR 2024 · similarity 0.84)
• Multi-Part Object Representations via Graph Structures and Co-Part Discovery (NeurIPS 2026 · similarity 0.84)