Research map / Vision-language and multimodal / Multimodal LLMs

Remember Me: Bridging the Long-Range Gap in LVLMs with Three-Step Inference-Only Decay Resilience Strategies

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes T-DRS, an inference-only, training-free three-step attention modification (semantic residual, distance-aware smoothing, and heavy-tailed reinforcement) that counteracts RoPE long-range attention decay in LVLMs, reporting small accuracy gains on VQA and POPE benchmarks.

Problem

RoPE in LVLMs causes progressive attention decay over long token distances, which weakens cross-modal long-range dependencies needed for VQA, and prior fixes typically require retraining.

Approach

Three plug-in attention-logit modifications applied at inference: a cosine-similarity semantic bias (SD-DRS), a Gaussian distance-aware decay modulated by semantic scale (DC-DRS), and a rational-quadratic heavy-tailed reinforcement term (reRD-DRS), summed residually with the original RoPE logits.

Key results

• Adding T-DRS to LLaVA-1.5-7B, InternVL2-8B and Qwen2.5-VL-7B gives small gains on ScienceQA-IMG (+1.3, +0.7, +1.3 points) and TextVQA (+0.8, +0.6, +0.5 points).
• On POPE, Qwen2.5-VL-7B with T-DRS reaches 88.5 accuracy versus 87.7 baseline; LLaVA-1.5-7B goes 83.3 to 83.7.
• Ablation shows each component adds gains, with the full model best on ScienceQA-IMG and POPE for all three backbones.

Limitations

• Gains are small (mostly under 1.5 points) and evaluated on a limited set of benchmarks.
• Hyperparameters (w_min, lambda) are chosen by sensitivity analysis on the evaluation sets themselves.
• Some baseline numbers appear to be taken from prior reports rather than re-run, and comparisons across methods are not controlled.

Experimental setup

Datasets: ScienceQA-IMG GQA TextVQA POPE

Benchmarks: ScienceQA-IMG GQA TextVQA POPE

Metrics: accuracy F1-score

Baselines: InstructBLIP-7B InstructBLIP-13B BLIP-2-13B Qwen-VL LLaVA1.5-7B InternVL2-8B Qwen2.5-VL-7B MobileVLM-v2-7B

Builds on: RoPE LLaVA-1.5 InternVL2 Qwen2.5-VL

Closest papers by meaning

• Mitigating Mask Prior Drift and Positional Attention Collapse in Large Diffusion Vision-Language Models (ICML 2026 · similarity 0.89)
• Large Vision-Language Models Get Lost in Attention (ICML 2026 · similarity 0.89)
• Unified Knowledge Maintenance Pruning and Progressive Recovery with Weight Recalling for Large Vision-Language Models (AAAI 2025 · similarity 0.89)
• Unveiling the Tapestry of Consistency in Large Vision-Language Models (NeurIPS 2024 · similarity 0.89)
• A-VL: Adaptive Attention for Large Vision-Language Models (AAAI 2025 · similarity 0.89)
• Robust-R1: Degradation-Aware Reasoning for Robust Visual Understanding (AAAI 2026 · similarity 0.89)