Research map / Vision-language and multimodal / Human and embodied modeling
Vision Mamba Mender
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Vision Mamba Mender, a post-hoc method that analyzes Mamba's external and internal hidden-state correlations (Grad-ESC/Grad-ISC) to find flaws and repairs them with correlation-constraint losses, improving several vision Mamba models on ImageNet subsets without extra parameters.
Problem
Vision Mamba models are optimized mainly by predefined architectural changes that need strong priors and trial and error; their internal working mechanisms in visual recognition are poorly understood, and it is unclear how to fix their flaws post hoc.
Approach
Grad-CAM-style gradient-weighted correlation scores for Mamba's external states (Grad-ESC, scored by perturbation and segmentation tests against foreground masks) and internal states (Grad-ISC, scored by simplicity and cross-sample homogeneity), used to detect flaws and then to repair them via added loss terms that constrain these correlations during fine-tuning.
Key results
- On ImageNet-50, combined external and internal repair raises ViM-T accuracy from 76.44 to 79.68 (+3.24).
- SiMBA-S gains the most on ImageNet-50, rising from 81.32 to 86.52 (+5.20) with all repairs.
- On ImageNet-1K, LocalVim-T improves from 57.70 to 60.83 (+3.13) with combined repair, and ViM-T reaches 73.30 (+1.66).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Repair external state flaws (+Ext) (ablation) | ViM-T · ImageNet-50 | accuracy: 76.44 -> 78.48 (+2.04) | Helped |
| Repair internal state flaws (+Int) (ablation) | ViM-T · ImageNet-50 | accuracy: 76.44 -> 78.20 (+1.76) | Helped |
| Repair both external and internal state flaws (+All) (ablation) | ViM-T · ImageNet-50 | accuracy: 76.44 -> 79.68 (+3.24) | Helped |
| Repair external state flaws (+Ext) (ablation) | VMamba-T · ImageNet-50 | accuracy: 79.96 -> 81.08 (+1.12) | Helped |
| Repair internal state flaws (+Int) (ablation) | VMamba-T · ImageNet-50 | accuracy: 79.96 -> 82.36 (+2.40) | Helped |
| Repair external state flaws (+Ext) (ablation) | SiMBA-S · ImageNet-50 | accuracy: 81.32 -> 84.48 (+3.16) | Helped |
| Repair internal state flaws (+Int) (ablation) | LocalViM-T · ImageNet-1K | accuracy: 57.70 -> 59.90 (+2.20) | Helped |
| Repair external flaws in c(ℓ)n and s(ℓ)n (ablation) | ViM last block · ImageNet-50 | accuracy: 76.44 -> 78.48 | Helped |
| Repair external flaw in x(ℓ)n (ablation) | ViM last block · ImageNet-50 | accuracy: 76.44 -> 78.40 | Helped |
| Repair external flaw in c(ℓ)n (ablation) | ViM last block · ImageNet-50 | accuracy: 76.44 -> 78.44 | Helped |
| Repair external flaw in s(ℓ)n (ablation) | ViM last block · ImageNet-50 | accuracy: 76.44 -> 78.28 | Helped |
| Repair external flaw in z(ℓ)n (ablation) | ViM last block · ImageNet-50 | accuracy: 76.44 -> 77.96 | Helped |
Surprising findings
- Repairing external flaws in the tenth block of ViM still improves accuracy (by 1.72%), indicating the effect is not confined to one block.
Limitations
- Experiments reduce model depth and hidden dimensions and use patch size 16, so absolute baselines are weaker than published numbers.
- Repair uses foreground annotations from ImageNet-S, limiting applicability to datasets without masks.
- Analysis only considers module-level states; internal SSM details were not examined.
Experimental setup
Datasets: ImageNet-50 ImageNet-300 ImageNet-1K ImageNet-S
Benchmarks: ImageNet-50 ImageNet-300 ImageNet-1K
Metrics: top-1 accuracy
Baselines: ViM-T VMamba-T SiMBA-S EfficientVMamba-T LocalVim-T
Builds on: Mamba Grad-CAM Vision Mamba (ViM) VMamba DeiT training recipe
Closest papers by meaning
- Demystify Mamba in Vision: A Linear Attention Perspective (NeurIPS 2024 · similarity 0.93)
- MambaOut: Do We Really Need Mamba for Vision? (CVPR 2025 · similarity 0.93)
- Mamba-Adaptor: State Space Model Adaptor for Visual Recognition (CVPR 2025 · similarity 0.92)
- Spatial-Mamba: Effective Visual State Space Models via Structure-Aware State Fusion (ICLR 2025 · similarity 0.90)
- Achilles' Heel of Mamba: Essential difficulties of the Mamba architecture demonstrated by synthetic data (NeurIPS 2025 · similarity 0.90)
- Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Model (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.