Research map / Vision-language and multimodal / Multimodal LLMs

Revisiting Visual Corruptions in LVLMs: A Shape-Texture Perspective on Model Failures

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Bias-aware multi-image robustness

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ST-CD, a training-free contrastive decoding method for LVLMs that uses edge-extracted (shape) and jigsaw-permuted (texture) contrastive views with entropy-based fusion to improve robustness to visual corruptions.

Problem

LVLMs are highly vulnerable to visual corruptions (noise, blur, geometric distortions, color shifts), and prior explanations treat corruption as monolithic, overlooking that different corruptions impair shape vs. texture perception differently.

Approach

Organizes corruptions along a shape-texture perceptual subspace, showing shape-degraded (blur, geometric) and texture-degraded (noise, color) corruptions cause opposite failure modes; then builds two complementary contrastive views (Canny edge and jigsaw permutation) and fuses their logit corrections with entropy-based adaptive weights.

Key results

• ST-CD improves average accuracy over baseline by 10.0% (LLaVA-1.5), 6.5% (Qwen-VL), and 8.4% (mPLUG-Owl2) on ImageNet10-C, and gains 3.2%, 3.1%, 4.7% over VCD on those architectures.
• ST-CD achieves about 1.5x inference overhead versus 4.2x for VACoDe.
• A learned fusion-weight variant reaches 86.3% average on ImageNet10-C with LLaVA-1.5 versus 84.1% for entropy weighting, but requires corrupted validation data.

Surprising findings

• Learned fusion weights outperform entropy weighting by over 2 points but need a small set of corrupted training data, which conflicts with the zero-shot framing.

Limitations

• Shape and texture probes are simple heuristics (Canny edges, jigsaw permutation) that may be weak at very low resolution.
• Best fusion variant requires corrupted validation data, which may be impractical in zero-shot settings.
• Evaluation focuses on a 10-class ImageNet subset and synthetic corruptions with only 8 corruption types.

Experimental setup

Datasets: ImageNet10-C MMBench-C POPE-MSCOCO-C RWIC-VQA

Benchmarks: ImageNet10-C MMBench-C POPE-MSCOCO-C RWIC-VQA

Metrics: accuracy

Baselines: VCD ICD LCD VACoDe Baseline (no decoding)

Builds on: Visual Contrastive Decoding ImageNet-C Shape-texture bias (Geirhos et al.)

Closest papers by meaning

• MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMs (ICML 2026 · similarity 0.89)
• VLM-RobustBench: A Comprehensive Benchmark for Robustness of Vision-Language Models (ICML 2026 · similarity 0.89)
• Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding? (ICML 2026 · similarity 0.88)
• Are We on the Right Way for Evaluating Large Vision-Language Models? (NeurIPS 2024 · similarity 0.88)
• Can We Talk Models Into Seeing the World Differently? (ICLR 2025 · similarity 0.88)
• Scaling Test-Time Robustness of Vision-Language Models via Self-Critical Inference Framework (CVPR 2026 · similarity 0.88)