Research map / Vision-language and multimodal / Multimodal LLMs

Disrupting Hierarchical Reasoning: Adversarial Protection for Geographic Privacy in Multimodal Reasoning Models

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ReasonBreak, a concept-aware adversarial generator that perturbs image blocks toward hard-negative concept embeddings to disrupt hierarchical geographic reasoning in multimodal reasoning models, and releases the GeoPrivacy-6K dataset.

Problem

Multimodal large reasoning models infer precise geographic locations from personal images via hierarchical chain-of-thought, and existing perception-oriented adversarial privacy defenses fail against this multi-step reasoning.

Approach

Decompose ultra-high-resolution images into blocks, assign each block its geographic concepts, pick a hard-negative concept prior from an embedding bank, and train a decoder conditioned on that prior to generate block-wise perturbations that minimize surrogate CLIP representation similarity, aiming to break conceptual and sequential dependencies in reasoning chains.

Key results

• Average Top-1 tract-level PPR of 33.8% vs 19.4% for the strongest baseline at epsilon=16/255 across seven MLRMs on DoxBench.
• Average Top-1 block-level PPR of 33.5% vs 16.8% for the best baseline.
• On Gemini 2.5 Pro Top-1 block-level, ReasonBreak achieves 23.3% PPR while baselines reach 0.0%.

Surprising findings

• On InternVL 3.0 72B, protection at tract and block levels is higher with epsilon=8 than epsilon=16 (inverted scaling).
• Adversarial perturbations occasionally improve MLRM accuracy (negative protection rates), and a single added line break in the prompt drastically changes InternVL outputs even at temperature 0.

Limitations

• Fails on images with dominant machine-readable text (OCR-based location cues), as only two DoxBench failure cases showed.
• Evaluation metric PPR and results depend on stochastic MLRM outputs; inverted scaling and negative protection rates indicate noisy evaluation.
• Relies on transfer from CLIP surrogate encoders to closed-source MLRMs without direct access.

Experimental setup

Datasets: GeoPrivacy-6K HoliCity Aesthetic-4K LHQ DoxBench

Benchmarks: DoxBench

Metrics: Privacy Protection Rate (Top-1) Privacy Protection Rate (Top-3)

Baselines: AnyAttack M-Attack

Builds on: AnyAttack M-Attack CLIP

Closest papers by meaning

• Do Vision-Language Models Respect Contextual Integrity in Location Disclosure? (ICLR 2026 · similarity 0.88)
• MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models (ICML 2026 · similarity 0.88)
• The Emotional Baby Is Truly Deadly: Does Your Multimodal Large Reasoning Model Have Emotional Flattery Towards Humans? (AAAI 2026 · similarity 0.85)
• STaR: Sensitive Trajectory Regulation for Unlearning in Large Reasoning Models (AAAI 2026 · similarity 0.84)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory (ICLR 2024 · similarity 0.84)
• Anchoring the Mind of Multimodal Reasoners: Cognitive Bias as a Vector for Jailbreak Attacks (CVPR 2026 · similarity 0.84)