Research map / Architectures and efficiency / Attention and transformers
MobileMamba: Lightweight Multi-Receptive Visual Mamba Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MobileMamba, a three-stage lightweight hybrid Mamba/CNN network with a multi-receptive field module (WTE-Mamba, MK-DeConv, identity) plus distillation and longer training, achieving high ImageNet-1K top-1 with high GPU throughput.
Problem
Lightweight vision models either have local receptive fields (CNNs) or quadratic attention cost (ViTs), and existing lightweight Mamba models have low FLOPs but poor real inference throughput and accuracy.
Approach
A three-stage network whose MRFFI module splits channels into a wavelet-enhanced bidirectional Mamba branch (global + high-frequency detail), a multi-kernel depthwise convolution branch, and an identity branch; combined with knowledge distillation, extended 1000-epoch training, and normalization-layer fusion at test time.
Key results
- MobileMamba reaches 83.6% Top-1 on ImageNet-1K at 4G-FLOPs scale (with training strategies), and 76.9/78.9/80.7/82.2/83.3 for the other scales.
- MobileMamba-S6 is about 21x faster than LocalVim on GPU and about 3.3x faster than EfficientVMamba in throughput comparisons, with +2.0 Top-1 over EfficientVMamba at the comparable scale.
- On ADE20K at 512x512 with PSPNet, MobileMamba-B4 reaches 36.9 mIoU using 8.5%-11.2% of the FLOPs of MobileNetv2 and MobileViTv2 (per the text).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add knowledge distillation (soft distillation from TResNet-L teacher) (ablation) | MobileMamba-S6 · ImageNet-1K | Top-1: +2.0 | Helped |
| extend training from 300 to 1000 epochs (ablation) | MobileMamba-S6 with KD · ImageNet-1K | Top-1: +0.7 | Helped |
| normalization layer fusion at test time (ablation) | MobileMamba-S6 · ImageNet-1K | Top-1 (accuracy unchanged; throughput 9200 to 11000): 0 Top-1 change | No clear effect |
| add wavelet transform (WTE-Mamba) branch (ablation) | MobileMamba-S6 · ImageNet-1K | Top-1: +0.2 (throughput drops 11687 to 11000) | Helped |
| split channels into 3 multi-kernel branches (3,5,7) in MK-DeConv (ablation) | MobileMamba-S6 · ImageNet-1K | Top-1: -0.1 (n=3 vs n=1) | No clear effect |
| use bidirectional scan with expand ratio 2 and d_state 1 instead of 4 directions, ratio 1, d_state 16 (ablation) | Mamba component in MobileMamba · ImageNet-1K | Top-1: +0.3 (78.0 vs 77.7) with higher throughput (11000 vs 9945) | Helped |
| increase input resolution for large models (224 to 384 and 512) with same architecture (ablation) | MobileMamba-B1 architecture · ImageNet-1K | Top-1: +1.6 (384) and +1.8 (512) | Helped |
| replace Mamba with TTT, xLSTM, or RWKV6 RNN-style blocks (ablation) | MobileMamba backbone · ImageNet-1K | Top-1: TTT 77.0, xLSTM 77.3, RWKV6 77.8 vs Mamba 78.0 | Hurt |
| three-stage network instead of four-stage (with CNN first two stages) (ablation) | MobileMamba coarse design · ImageNet-1K | Top-1 and Top-5 at similar throughput: +0.4 Top-1 and Top-5 | Helped |
| training strategies (KD + extended epochs) applied to small models | MobileMamba-T2 · ImageNet-1K | Top-1: +3.3 (73.6 to 76.9) | Helped |
| MobileMamba vs EfficientVMamba-S/T style Mamba baselines (architecture) | MobileMamba-S6 vs EfficientVMamba-T · ImageNet-1K | Top-1: +1.5 at 18.5% fewer FLOPs | Helped |
| MobileMamba-B1 vs EMO-5M on Mask R-CNN backbone | MobileMamba-B1 detection backbone · COCO Mask R-CNN | mAP b / mAP m: +1.3 mAP b, +1.0 mAP m | Helped |
Surprising findings
- Under similar FLOPs, a three-stage network outperforms four-stage networks in both accuracy and throughput, contrary to the common four-stage design.
- Current Mamba models are slower than Transformers on CPU and mobile latency, despite low FLOPs.
Limitations
- CPU and mobile edge latency lag Transformer-based lightweight models due to immature Mamba engineering.
- Training strategies (distillation, 1000 epochs) are applied only to some variants, so headline numbers mix training recipes and are not directly comparable to baselines trained for 300 epochs.
Experimental setup
Datasets: ImageNet-1K MS-COCO 2017 ADE20K
Benchmarks: ImageNet-1K MS-COCO 2017 ADE20K
Metrics: Top-1 accuracy Top-5 accuracy mAP mIoU GPU throughput CPU throughput latency FLOPs
Baselines: EfficientVMamba LocalVim SHViT EMO EfficientViT VRWKV MobileNetv2 MobileViTv2
Builds on: Mamba EfficientViT DeiT knowledge distillation VMamba Vim
Closest papers by meaning
- EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba (AAAI 2025 · similarity 0.90)
- Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Model (NeurIPS 2024 · similarity 0.90)
- Demystify Mamba in Vision: A Linear Attention Perspective (NeurIPS 2024 · similarity 0.89)
- MambaOut: Do We Really Need Mamba for Vision? (CVPR 2025 · similarity 0.89)
- Vision Mamba Mender (NeurIPS 2024 · similarity 0.89)
- DiMSUM: Diffusion Mamba - A Scalable and Unified Spatial-Frequency Method for Image Generation (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.