Research map / Vision-language and multimodal / Multimodal LLMs

LLMind: Bio-inspired Training-free Adaptive Visual Representations for Vision-Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LLMind, a training-free bio-inspired adaptive sampler that uses a Möbius warp with test-time SPSA-based semantic feedback to allocate limited pixels to salient regions for frozen VLMs.

Problem

VLMs sample images uniformly at fixed resolution, wasting pixels on uninformative regions and degrading badly under tight pixel budgets; existing foveated/bio-inspired samplers are task-specific, trained, or require white-box access.

Approach

A Möbius-transformation-based sampling front-end (BASS) that magnifies salient regions and compresses the periphery, with its 4 parameters optimized at inference via a perceptual loss plus a gradient-free SPSA loop driven by the frozen VLM's answer embedding similarity (CSF), so it works with black-box VLMs.

Key results

• On A-OKVQA with Qwen2.5-VL, LLMind reaches 69.74% at 5% pixels vs 52.01% for uniform sampling (+34.09%) and 85.67% full-resolution.
• LLMind retains up to 97% of full-resolution accuracy at 5% pixels on A-OKVQA (SmolVLM, 76.81 vs 78.60).
• On LVIS single-region VQA with Qwen3-VL, LLMind at 3% pixels reaches 62.90% vs 50.60% full resolution baseline; uniform sampling gets 45.63%.

Surprising findings

• On region-guided LVIS VQA, LLMind with 3% pixels outperforms full-resolution input for small VLMs.
• Static foveated and radial baselines often underperform uniform sampling, and LLaVA-OneVision with several baselines drops sharply.

Limitations

• SPSA-based optimization adds substantial per-iteration latency (~4.75 s/iter).
• Evaluation restricted to lightweight VLMs under 4B parameters.
• Sampling-based gains are reported only on pixel budgets of 1-10% and on a subset of benchmarks.

Experimental setup

Datasets: VQAv2 Seed-Bench A-OKVQA LVIS

Benchmarks: VQAv2 Seed-Bench A-OKVQA LVIS region-guided VQA

Metrics: accuracy accuracy gain over uniform sampling accuracy retained relative to full resolution

Baselines: Uniform Sampling Static Foveated Sunflower Inspired Radial Sampling

Builds on: SPSA ControlMLLM information-matched images (Gizdov et al.) sunflower phyllotaxis sampling (Killick et al.)

Closest papers by meaning

• AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisition (CVPR 2026 · similarity 0.90)
• Frustratingly Easy Test-Time Adaptation of Vision-Language Models (NeurIPS 2024 · similarity 0.90)
• Language-Instructed Vision Embeddings for Controllable and Generalizable Perception (ICLR 2026 · similarity 0.89)
• Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models (ICML 2024 · similarity 0.89)
• NeuronEye: Query-Guided Visual Concept Activation for Vision-Language Reasoning (NeurIPS 2026 · similarity 0.89)
• Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training (ICLR 2026 · similarity 0.89)