Research map / Architectures and efficiency / Attention and transformers
Boosting Vision State Space Model with Fractal Scanning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FractalMamba, a vision state space model that serializes image patches with Hilbert fractal scanning curves and adds a Fractal Fusion Pathway (bidirectional state transitions plus a one-pixel shift) to avoid the cost of multi-direction scanning.
Problem
Linear scanning curves (row-by-row, zigzag, Z-order) used to serialize image patches for vision Mambas lose 2D spatial adjacency and produce repetitive patterns, hurting accuracy and scale robustness; multi-direction scanning fixes this but increases computation.
Approach
Replace linear scanning with Hilbert fractal scanning curves, which preserve spatial proximity and relative distances across resolutions, and add a Fractal Fusion Pathway that makes SSM state transitions bidirectional with a memory mechanism and a one-pixel curve shift, giving multi-direction-like benefits with a single curve.
Key results
- FractalMamba-T reaches 83.0% top-1 on ImageNet-1K at 224 resolution with 31M params and 4.8G FLOPs, above VMamba-T (82.5) and Swin-T (81.3).
- On COCO Mask R-CNN 1x schedule, FractalMamba-T achieves 46.8 box AP and 42.4 mask AP, slightly above VMamba-T (46.5 / 42.1).
- On LEVIR-CD+ change detection at 1024x1024, FractalMamba-T achieves 80.0 IoU and 89.9 F1 versus ChangeMamba-B's 79.2 IoU and 88.4 F1, with fewer FLOPs (55G vs 179G).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace linear scanning with fractal scanning (ablation) | ViM · ImageNet-1K, 224²–1024² | Top-1 accuracy: 76.1 -> 77.9; 70.4 -> 75.1; 67.4 -> 73.8; 51.4 -> 62.3; 30.6 -> 51.3; 16.1 -> 39.7 | Helped |
| replace linear scanning with fractal scanning, using 2 curves instead of 4 (ablation) | VMamba · ImageNet-1K, 224²–1024² | Top-1 accuracy: 82.5 -> 82.7; 82.5 -> 82.8; 81.1 -> 82.0; 79.3 -> 80.6; 76.1 -> 78.1; 62.3 -> 72.3 | Helped |
| replace linear scanning with fractal scanning (ablation) | VMamba · ImageNet-1K, 224²–1024² | Top-1 accuracy: 82.5 -> 82.9; 82.5 -> 83.3; 81.1 -> 82.5; 79.3 -> 81.6; 76.1 -> 79.7; 62.3 -> 74.9 | Helped |
| add the shifting operation to FFP (ablation) | FractalMamba · ImageNet-1K, 224²–1024² | Top-1 accuracy: 82.8 -> 83.0; 83.0 -> 83.9; 82.0 -> 83.0; 81.1 -> 81.8; 79.6 -> 80.3; 74.7 -> 76.3 | Helped |
Surprising findings
- A single fractal curve with FFP (no shifting) outperforms VMamba with two linear curves at several resolutions, and matches four-curve VMamba at some resolutions.
Limitations
- Using a single Hilbert curve still loses some information for diagonal adjacency, as the authors note.
- Only the Hilbert curve is studied; other fractal curves are left to future work.
- Gains on COCO detection over VMamba-T are small (e.g., 46.8 vs 46.5 box AP at 1x).
Experimental setup
Datasets: ImageNet-1K MSCOCO 2017 LEVIR-CD+ ADE20K
Benchmarks: ImageNet-1K MSCOCO 2017 LEVIR-CD+ ADE20K
Metrics: Top-1 accuracy box AP mask AP IoU F1 Kappa mIoU
Baselines: ViM VMamba LocalMamba PlainMamba Swin Transformer ConvNeXt DeiT ChangeMamba RegNetY ResNet
Builds on: Mamba VMamba Vision Mamba (ViM) LocalMamba PlainMamba
Closest papers by meaning
- Spatial-Mamba: Effective Visual State Space Models via Structure-Aware State Fusion (ICLR 2025 · similarity 0.88)
- Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Model (NeurIPS 2024 · similarity 0.88)
- Vision Mamba Mender (NeurIPS 2024 · similarity 0.87)
- 2DMamba: Efficient State Space Model for Image Representation with Applications on Giga-Pixel Whole Slide Image Classification (CVPR 2025 · similarity 0.86)
- EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba (AAAI 2025 · similarity 0.86)
- GroupMamba: Efficient Group-Based Visual State Space Model (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.