Research map / Architectures and efficiency / Attention and transformers
Mamba architectures
Analyzes and adapts Mamba selective state space models, especially for vision tasks.
40 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2024–2026) · an approach in Attention and transformers, keyword cluster “sequence · long · modeling”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
6 papers in 2023–24 and 34 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 1.0% to 2.9%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2024 · MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection (NeurIPS 2024 · 69 citations)
- 2025 · MambaOut: Do We Really Need Mamba for Vision? (CVPR 2025 · 188 citations)
- 2026 · Mamba-3: Improved Sequence Modeling using State Space Principles (ICLR 2026 · 1 citation)
Earliest papers here
- MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection (NeurIPS 2024 · 69 citations)
- Demystify Mamba in Vision: A Linear Attention Perspective (NeurIPS 2024 · 58 citations)
- Decision Mamba: Reinforcement Learning via Hybrid Selective Sequence Modeling (NeurIPS 2024 · 43 citations)
What has been tried
29 papers here report 314 measured changes (a component added, removed or swapped, compared with a baseline): helped 72%, no clear effect 4%, hurt 21%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| replace HSM-SSD with NC-SSD | ImageNet-1K · Top-1 accuracy | 76.2 -> 75.4 | Hurt | EfficientViM: Efficient Vision Mamba with Hidden State Mixer based State Space Duality (CVPR 2025 · 69 citations) |
| add input gate | ImageNet-1K val · Top-1 | 77.6 -> 77.8 | Helped | Demystify Mamba in Vision: A Linear Attention Perspective (NeurIPS 2024 · 58 citations) |
| Add valuable sub-goals | Grid World · mean episode return | Helped | Decision Mamba: Reinforcement Learning via Hybrid Selective Sequence Modeling (NeurIPS 2024 · 43 citations) | |
| Use a learnable non-tissue padding token instead of fixed-zero padding | PANDA · accuracy | 0.4868 -> 0.5075 | Helped | 2DMamba: Efficient State Space Model for Image Representation with Applications on Giga-Pixel Whole Slide Image Classification (CVPR 2025 · 36 citations) |
| Add PF-Mamba with partial flipping | Beauty · HR@10 | 0.0953 -> 0.0986 | Helped | SIGMA: Selective Gated Mamba for Sequential Recommendation (AAAI 2025 · 25 citations) |
| replace joint scan with sequential scan | MegaDepth (15-epoch ablation) · AUC@5° | 64.5 -> 62.2 | Hurt | JamMa: Ultra-lightweight Local Feature Matching with Joint Mamba (CVPR 2025 · 24 citations) |
| Replace baseline linear-attention blocks with LAMA | Rain100L · PSNR | 38.24 -> 38.89 | Helped | ACL: Activating Capability of Linear Attention for Image Restoration (CVPR 2025 · 21 citations) |
| add channel-wise autoregressive modeling | Kodak · BD-rate | -6.73% -> -12.52% | Helped | MambaIC: State Space Models for High-Performance Learned Image Compression (CVPR 2025 · 21 citations) |
| Add BiMamba to the Mamba baseline | BRACS · accuracy | 0.540 -> 0.586 | Helped | M3amba: Memory Mamba is All You Need for Whole Slide Image Classification (CVPR 2025 · 12 citations) |
| use height-prioritized 2D Hilbert expansion instead of XYZ sequence reordering | OpenOccupancy validation v0.0 · mIoU | 24.5 -> 25.2 | Helped | OccMamba: Semantic Occupancy Prediction with State Space Models (CVPR 2025 · 11 citations) |
| Remove Mamba | MIPI · PSNR (dB) | 36.95 -> 35.17 | Hurt | Binarized Mamba-Transformer for Lightweight Quad Bayer HybridEVS Demosaicing (CVPR 2025 · 7 citations) |
| add the illumination-restoration branch M_L | SICE average · PSNR | 21.55 -> 22.05 | Helped | ECMamba: Consolidating Selective State Space Model with Retinex Guidance for Efficient Multiple Exposure Correction (NeurIPS 2024 · 6 citations) |
What reviewers will expect
Datasets and baselines named in the 26 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ImageNet-1K · 10 of 26 papers
- ADE20K · 8 of 26 papers
- COCO 2017 · 6 of 26 papers
- TCGA-BRCA · 2 of 26 papers
- BRACS · 2 of 26 papers
- PG19 · 2 of 26 papers
- Pile · 2 of 26 papers
- WikiText-103 · 2 of 26 papers
Baselines
- Vim · 6 of 26 papers
- VMamba · 6 of 26 papers
- ConvNeXt · 4 of 26 papers
- PlainMamba · 2 of 26 papers
- Swin Transformer · 2 of 26 papers
- DeiT · 2 of 26 papers
- ResNet · 2 of 26 papers
- AB-MIL · 2 of 26 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 40 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 4 papers
- Shanghai Jiao Tong University · 3 papers
- Jilin University · 2 papers
- Korea University · 2 papers
- Harvard University · 2 papers
- Harbin Institute of Technology · 2 papers
- Korea Advanced Institute of Science and Technology · 2 papers
- Southeast University · 2 papers
Use it
Download BibTeX of 20 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- State space and recurrent sequence models (172 papers)
- Sparse efficient transformers for long context (76 papers)
- Linear attention approximations (76 papers)
Elsewhere in the topic
- Transformer expressivity and learnability (217 papers)
- Sparse and structured attention (163 papers)
- Sparse attention for long-context LLMs (147 papers)
- Attention head and block redesign (127 papers)
- Efficient and robust vision transformers (124 papers)
- Efficient language transformers (105 papers)
- Language model pretraining and induction (99 papers)
- Hybrid and interpretable ViT designs (94 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).