Research map / Vision-language and multimodal / Multimodal LLMs

Do Vision-Language Models Respect Contextual Integrity in Location Disclosure?

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces VLM-GeoPrivacy, a 1,200-image benchmark testing whether VLMs judge context-appropriate location disclosure (contextual integrity), finding 14 VLMs over-disclose and are vulnerable to iterative and FigStep-style malicious prompting.

Problem

Vision-language models can geolocate casually shared photos at street-level precision, risking privacy violations; existing guardrails use blanket granularity restrictions that ignore image context and sharer intent.

Approach

Extends contextual integrity to multimodal geolocation: real-world images annotated by humans with visual distinctiveness, sharing intent, subject visibility, and expected disclosure granularity, used both for MCQ judgment and free-form geolocation under vanilla, iterative CoT, and FigStep-based malicious prompting.

Key results

• Gemini-2.5 Flash reaches 82.0% context accuracy and 66.6% granularity accuracy on MCQ, while the best free-form model (o3) matches human granularity only 49.7% of the time.
• GPT-5 over-discloses image location 47.6% of the time under vanilla geolocation queries.
• o3 exposes location over 97% of the time under iterative CoT prompting, with over 30% of cases within 1 km.

Surprising findings

• Smaller GPT-4.1-mini often outperforms the larger GPT-4.1 on privacy judgment, an inverse-scaling pattern, attributed to more frequent abstention.
• Claude Sonnet 4 has the best privacy preservation but under-discloses heavily, yielding low geolocation utility.

Limitations

• Evaluates only one malicious prompting method (FigStep); more attacks combining text and image are not explored.
• Targets perceived user intent and average human expectations rather than individual or cultural variation in sharing norms.

Experimental setup

Datasets: YFCC100M IM2GPS-3k GPTGeoChat

Benchmarks: VLM-GeoPrivacy

Metrics: accuracy F1 over-disclosure rate under-disclosure rate contextualized location exposure rate abstention violation rate street/city/region geolocation accuracy

Baselines: Random Llama-3.2-11B-Vision Llama-3.2-90B-Vision Qwen2.5-VL-7B Qwen2.5-VL-72B Deepseek-VL2

Builds on: contextual integrity theory FigStep chain-of-thought least-to-most prompting

Closest papers by meaning

• Disrupting Hierarchical Reasoning: Adversarial Protection for Geographic Privacy in Multimodal Reasoning Models (ICLR 2026 · similarity 0.88)
• MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models (ICML 2026 · similarity 0.86)
• Private Attribute Inference from Images with Vision-Language Models (NeurIPS 2024 · similarity 0.86)
• Textual Supervision Enhances Geospatial Representations in Vision-Language Models (ICML 2026 · similarity 0.85)
• Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Models (NeurIPS 2025 · similarity 0.85)
• MMLongCite: A Benchmark for Evaluating Faithfulness of Long-Context Vision-Language Models (NeurIPS 2026 · similarity 0.85)