Research map / Vision-language and multimodal / Multimodal LLMs
VLM in a flash: I/O-Efficient Sparsification of Vision-Language Model via Neuron Chunking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Neuron Chunking, an I/O-aware activation sparsification method that selects contiguous neuron chunks by importance-per-flash-latency utility, speeding up flash-offloaded VLM inference on Jetson devices.
Problem
Activation sparsification for flash-offloaded VLM inference on edge devices assumes I/O latency scales linearly with data volume, but flash latency depends strongly on access contiguity, so magnitude-only selection yields fragmented reads and high I/O overhead.
Approach
Couple neuron importance with flash access cost: profile a per-chunk-size latency lookup table, then greedily select contiguous chunks ranked by importance divided by estimated latency (utility), plus an offline hot-cold neuron reordering step.
Key results
- I/O latency reduced by an average 2.19x on Jetson Orin Nano and up to 4.65x at comparable accuracy.
- On Jetson Orin AGX, average 2.89x speedup and up to 5.76x I/O latency reduction over top-k baseline.
- For LLaVA-7B on Nano, hot-cold reordering gives up to 1.23x speedup, rising to up to 2.55x with chunk selection added.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use contiguity-aware Neuron Chunking instead of top-k activation sparsification | Flash-offloaded VLM inference · All evaluated models and datasets, Jetson Orin Nano | I/O speedup: 2.19× average; up to 4.65× | Helped |
| Use contiguity-aware Neuron Chunking instead of top-k activation sparsification | Flash-offloaded VLM inference · All evaluated models and datasets, Jetson Orin AGX | I/O speedup: 2.89× average; up to 5.76× | Helped |
| Add hot-cold reordering (ablation) | Baseline activation sparsification on LLaVA-OneVision-7B · LLaVA-OneVision-7B, Jetson Orin Nano | Speedup vs. baseline: up to 1.23× | Helped |
| Add chunk selection to hot-cold reordering (ablation) | Baseline activation sparsification on LLaVA-OneVision-7B · LLaVA-OneVision-7B, Jetson Orin Nano | Speedup vs. baseline: up to 1.23× -> up to 2.55× | Helped |
Surprising findings
- Higher activation sparsity can increase flash read latency in VLMs because fragmented reads degrade throughput more than data volume is reduced.
- Hot-cold reordering alone yields only modest gains; most improvement comes from online chunk selection.
Limitations
- Greedy chunk selection ignores synergies among adjacent low-scoring chunks.
- Latency model is near-linear with a proportional bias and weakens for smaller models or lower-end devices.
- Evaluation limited to frame-by-frame VLMs and two Jetson devices; no full hyperparameter sensitivity analysis.
Experimental setup
Datasets: TempCompass NExT-QA VideoDetailCaption
Benchmarks: TempCompass NExT-QA VideoDetailCaption
Metrics: accuracy I/O latency end-to-end latency speedup at matched accuracy
Baselines: top-k magnitude activation sparsification (TEAL-style) LLM in a Flash bundling hot-cold reordering only
Builds on: TEAL CATS LLM in a Flash Ripple Deja Vu
Closest papers by meaning
- CoreMatching: A Co-adaptive Sparse Inference Framework with Token and Neuron Pruning for Comprehensive Acceleration of Vision-Language Models (ICML 2025 · similarity 0.91)
- The Sparsity Whisperer (NeurIPS 2026 · similarity 0.87)
- SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference (ICML 2025 · similarity 0.87)
- VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration (ICLR 2025 · similarity 0.87)
- NeuronEye: Query-Guided Visual Concept Activation for Vision-Language Reasoning (NeurIPS 2026 · similarity 0.87)
- EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.