Research map / Vision-language and multimodal / Robot action and navigation

Event-Grounded Sparse Autoencoders for Vision-Language-Action Policies

NeurIPS 2026 · Vision-language and multimodal › Robot action and navigation · approach: Vision-language-action foundation models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an event-grounded SAE pipeline that ties VLA SAE features to closed-loop behavioral events (AWE keyframes, task-local clustering) and validates them via residual-preserving latent zero-out interventions on OpenVLA and pi0.5 in LIBERO and on a real robot.

Problem

Mechanistic interpretability tools built for LLMs/VLMs (logit lens, text-context feature naming) do not transfer to VLAs, whose outputs are robot actions and whose interventions can only be validated via expensive closed-loop rollouts.

Approach

Anchor SAE feature ranking to external, SAE-independent behavioral events extracted from rollouts (kinematic keyframes via Automatic Waypoint Extraction, clustered with visual, state and temporal cues), score every alive feature against these events with an event-aligned template score, and causally test top features with residual-preserving latent edits.

Key results

• On OpenVLA layer 31, event-aligned zero-out of top-K features drops SR from 70.0% to 48.8% (-21.2 pp), versus -1.3 pp for random-alive features.
• On pi0.5 action expert, zero-out collapses SR to near 0% for event-aligned and other top-ranked features at layer 0, 5, 11, 17, while random-alive features cause smaller drops at most layers.
• pi0.5 PaliGemma backbone is largely robust to single-feature zero-out (SR changes within about 3 pp of 96.8% baseline).

Surprising findings

• On pi0.5 the action expert collapses under nearly any ranking, so ranking strategies are poorly distinguished there, with random-alive features also causing large drops at some layers.
• Target and off-target zero-out SR are nearly identical (0.641 vs 0.631), suggesting event-selected OpenVLA features are shared across tasks rather than task-specific.

Limitations

• Single-feature interventions and binary success rate cannot isolate one behavioral factor; selected features are shared and polysemantic.
• VLM semantic labels are used only for visualization, not for ranking or intervention scoring.
• Strong edits destabilize the real robot, so only conservative interventions were tested on hardware.

Experimental setup

Datasets: LIBERO-Spatial LIBERO-Object LIBERO-Goal LIBERO-10

Benchmarks: LIBERO-Spatial LIBERO-Object LIBERO-Goal LIBERO-10

Metrics: success rate delta SR fraction of variance explained alive fraction average L0 top-K overlap

Baselines: window-mean ranking task-mean ranking random-alive ranking FFN value-vector steering

Builds on: BatchTopK SAE Automatic Waypoint Extraction OpenVLA pi0.5 logit lens

Closest papers by meaning

• FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies (NeurIPS 2026 · similarity 0.90)
• NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
• OTTER: A Vision-Language-Action Model with Text-Aware Visual Feature Extraction (ICML 2025 · similarity 0.90)
• Beyond Data Scaling: Representation-Centric Pre-training for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.89)
• Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models (NeurIPS 2026 · similarity 0.89)
• ATI-VLA: Action-Centric Predictive Vision–Language–Action Models via Actionable Alignment Then Adaptive Injection (NeurIPS 2026 · similarity 0.89)