Research map / Vision-language and multimodal / Multimodal LLMs

LingoLoop Attack: Trapping MLLMs via Linguistic Context and State Entrapment into Endless Loops

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Training-free token reduction for MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LingoLoop, a white-box adversarial image attack that combines POS-aware EOS suppression with hidden-state norm constraints to trap MLLMs in repetitive, extremely verbose generation.

Problem

Energy-latency (sponge) attacks on multimodal LLMs aim to inflate output length and resource use, but prior methods apply uniform EOS suppression and ignore POS-dependent termination and repetition dynamics.

Approach

Weight EOS suppression per token by the POS tag of the preceding token (from a statistical prior pool), and add a loss penalizing L2 norms of generated-token hidden states to induce state collapse and looping, balanced dynamically and optimized with PGD.

Key results

• On Qwen2.5-VL-3B with MS-COCO, LingoLoop yields 1020.38 tokens vs 66.64 for clean inputs and 394.74 for Verbose Images, with 7090.58 J energy.
• With max_new_tokens relaxed to 32K, examples crafted at 2048 tokens produce 23844.90 tokens on MS-COCO (351.9x clean) and 23054.76 on ImageNet (367.6x).
• On InstructBLIP (MS-COCO), LingoLoop generates 1002.08 tokens versus 332.29 for Verbose Images.

Surprising findings

• Higher repetition_penalty (1.15) increased token counts for clean and Verbose Images inputs, and no_repeat_ngram_size=2 increased clean-input length, suggesting such defaults can lengthen outputs.

Limitations

• White-box threat model; transfer to black-box commercial models is only argued, not thoroughly demonstrated
• Relies on a statistical POS-EOS prior pool built per model from large image sets
• Evaluated on a small set of 100-200 images per dataset and four models

Experimental setup

Datasets: MS-COCO ImageNet

Benchmarks: MS-COCO ImageNet

Metrics: generated token count energy (J) latency (s)

Baselines: Verbose Images Noise None (clean inputs)

Builds on: Verbose Images projected gradient descent

Closest papers by meaning

• BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.87)
• Many-shot Jailbreaking (NeurIPS 2024 · similarity 0.87)
• MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models (NeurIPS 2026 · similarity 0.87)
• Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images (ICLR 2024 · similarity 0.86)
• LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs (ICLR 2025 · similarity 0.86)
• Inference-Time Vulnerability Beyond Shallow Safety: Alignment Along Generation Trajectories (NeurIPS 2026 · similarity 0.86)