Research map / Vision-language and multimodal / Multimodal LLMs

Visual Persuasion: What Influences Decisions of Vision-Language Models?

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies visual sensitivities of VLM agents in image-choice tasks by optimizing naturalistic, identity-preserving image edits (via an image editor and LLM/VLM feedback loops) and measuring how they shift choice probabilities, with an auto-interpretability pipeline and human validation.

Problem

VLM-based agents make consequential visual decisions (product purchase, hiring, housing, hotel choice), but their visual preferences and sensitivities to presentation changes are poorly understood beyond accuracy benchmarks.

Approach

Treat the VLM's decision function as a latent visual utility probed via revealed preference in head-to-head choices between systematically edited images; adapt text prompt optimizers (TextGrad, Feedback Descent) to visual prompt optimization and introduce CVPO, a competition-based method where judge panels select between two candidate images, then interpret discovered edits with a hierarchical aut

Key results

• Zero-shot edits raise selection probability by roughly 0.2-0.4 over original images across tasks, with all contrasts significant.
• CVPO's final images are chosen with probability 0.726-0.771 against VTG on most models, and beat VFD on 7 of 9 models by 0.04-0.21; VTG is far behind (0.46-0.64 gap).
• Human study (N=154) shows optimized images are chosen more often than originals (VTG 0.615 final vs 0.301 original for the strategy-level comparison); image normalization with 3 passes partially reduces sensitivity but does not eliminate it.

Surprising findings

• Distilling discovered visual themes into zero-shot editing instructions underperforms naive zero-shot edits for products, attributed to heterogeneity across product categories.
• Distinct optimization methods converge on overlapping visual themes, suggesting stable evaluator preferences.

Limitations

• Identity preservation is fuzzy; attribute changes may legitimately increase utility, confounding 'visual bias' with rational choice.
• Human study lacks statistical power for small head-to-head differences; no post-hoc contrasts significant.
• Distillation uses the same images as optimization; causal/mechanistic explanations not established.

Experimental setup

Datasets: Amazon Berkeley Objects (ABO) house price estimation dataset StyleGAN-Human (synthetic people) hotel image dataset from prior aesthetics work

Benchmarks: Product purchasing House searching Candidate hiring Hotel scouting

Metrics: choice probability in pairwise comparisons estimated marginal means from linear probability models order inconsistency rate iterations budget use

Baselines: VisualTextGrad VisualFeedbackDescent Original images Zero-shot edits

Builds on: TextGrad Feedback Descent Bradley-Terry model Matryoshka summarization

Closest papers by meaning

• GeoReward: Mitigating Contextual Variable Overestimation in Vision-Language Models for Cross-Market Preference Prediction (ICML 2026 · similarity 0.87)
• ARM-Thinker: Reinforcing Multimodal Generative Reward Models with Agentic Tool Use and Visual Reasoning (CVPR 2026 · similarity 0.87)
• Multimodal Large Language Models Make Text-to-Image Generative Models Align Better (NeurIPS 2024 · similarity 0.87)
• RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data (AAAI 2025 · similarity 0.86)
• UniAR: A Unified model for predicting human Attention and Responses on visual content (NeurIPS 2024 · similarity 0.86)
• Preference Adaptive and Sequential Text-to-Image Generation (ICML 2025 · similarity 0.86)