Research map / Architectures and efficiency / Attention and transformers
MambaOut: Do We Really Need Mamba for Vision?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Argues conceptually that Mamba's SSM suits long-sequence and autoregressive tasks, and empirically shows via MambaOut (Gated CNN blocks without SSM) that SSM is unnecessary for ImageNet classification but still lags visual Mamba models on COCO detection and ADE20K segmentation.
Problem
Visual Mamba models built with state space model token mixers underperform convolutional and attention-based models on vision tasks; it is unclear whether Mamba's SSM is needed for vision.
Approach
Characterizes Mamba's SSM as an RNN-like causal token mixer suited to long-sequence and autoregressive tasks, then tests this by stacking Gated CNN blocks (Mamba blocks with the SSM removed) into MambaOut models and comparing against visual Mamba models on ImageNet, COCO, and ADE20K.
Key results
- MambaOut-Small reaches 84.1% ImageNet top-1, 0.4% above LocalVMamba-S while using about 79% of its MACs.
- MambaOut-Tiny with Mask R-CNN trails VMamba-T by 1.4 APb and 1.1 APm on COCO.
- On ADE20K, LocalVMamba-T beats MambaOut-Tiny by 0.5 mIoU in both single- and multi-scale evaluation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace visual Mamba with MambaOut (remove SSM) | ImageNet image classification · ImageNet val, 224² (Small) | top-1 accuracy: 83.7 -> 84.1 | Helped |
| Replace VMamba-T with MambaOut-Tiny (remove SSM) | Mask R-CNN · COCO 2017 val, 1× schedule | APb: 46.5 -> 45.1 | Hurt |
| Replace VMamba-T with MambaOut-Tiny (remove SSM) | Mask R-CNN · COCO 2017 val, 1× schedule | APm: 42.1 -> 41.0 | Hurt |
| Replace LocalVMamba-T with MambaOut-Tiny (remove SSM) | UperNet · ADE20K val | mIoU (SS): 47.9 -> 47.4 | Hurt |
| Replace LocalVMamba-T with MambaOut-Tiny (remove SSM) | UperNet · ADE20K val | mIoU (MS): 49.1 -> 48.6 | Hurt |
Surprising findings
- Converting ViT attention from fully-visible to causal mode lowers ImageNet accuracy (e.g., ViT-S 79.8% to 78.9%), suggesting causal mixing is unnecessary for understanding tasks.
Limitations
- Only common vision tasks (ImageNet, COCO, ADE20K) were studied; medical, remote sensing, and language tasks were not explored.
- Scaling MambaOut beyond the reported sizes was not explored.
- The conceptual hypotheses are argued from FLOP-based length metrics and are not exhaustively validated.
Experimental setup
Datasets: ImageNet-1K COCO 2017 ADE20K
Benchmarks: ImageNet-1K COCO 2017 ADE20K
Metrics: top-1 accuracy APb APm mIoU MACs parameters
Baselines: Vision Mamba VMamba LocalVMamba PlainMamba EfficientVMamba ConvNeXt CAFormer TransNeXt
Builds on: Mamba Gated CNN MetaFormer ConvNeXt DeiT
Closest papers by meaning
- Vision Mamba Mender (NeurIPS 2024 · similarity 0.93)
- Demystify Mamba in Vision: A Linear Attention Perspective (NeurIPS 2024 · similarity 0.92)
- Autoregressive Pretraining with Mamba in Vision (ICLR 2025 · similarity 0.91)
- Mamba-Reg: Vision Mamba Also Needs Registers (CVPR 2025 · similarity 0.91)
- Mamba-Adaptor: State Space Model Adaptor for Visual Recognition (CVPR 2025 · similarity 0.91)
- MambaVision: A Hybrid Mamba-Transformer Vision Backbone (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.