Research map / Architectures and efficiency / Attention and transformers
Parallel Sequence Modeling via Generalized Spatial Propagation Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GSPN, a vision attention alternative that performs stable 2D line-scan linear propagation with row-stochastic weights, evaluated on ImageNet classification, DiT-style class-conditional generation, and SD text-to-image generation up to 16K.
Problem
Existing sub-quadratic attention models (linear attention, Mamba-style SSMs) flatten 2D images into 1D sequences, losing spatial coherence, while naive 2D linear propagation is unstable or has limited receptive field.
Approach
A 2D linear propagation layer whose propagation weights are made row-stochastic (sigmoid then row normalization over a tridiagonal three-neighbor connection) under a Stability-Context Condition, giving stable dense long-range context with line-scans in 4 directions, reducing effective sequence length to sqrt(N), and dropping positional embeddings.
Key results
- GSPN-T reaches 83.0% ImageNet top-1 at 5.3 GFLOPs, above VMamba-T (82.2%) and LocalVMamba-T (82.7%).
- GSPN-XL/2 reaches FID 15.26 on ImageNet 256x256 class-conditional generation, compared with DiT-XL/2 at 20.05, using 690M parameters.
- On COCO at 1024x1024, SD-v1.5-GSPN achieves FID 30.86 and CLIP-T 0.307 versus SD-v1.5 baseline 32.71 and 0.290; SD-XL with GSPN is reported up to 84x faster at 16K generation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GSPN-T instead of the strongest listed tiny-scale baseline | ImageNet classification · ImageNet-1K 224² | Top-1 accuracy: 82.7 -> 83.0 | Helped |
| use GSPN-B instead of the strongest listed base-scale baseline | ImageNet classification · ImageNet-1K 224² | Top-1 accuracy: 84.2 -> 84.3 | Helped |
| use GSPN-XL/2 instead of SiT-XL/2 | class-conditional generation · ImageNet 256×256 | FID: 18.04 -> 15.26 | Helped |
| replace SD-v1.5 attention with initialized GSPN | SD-v1.5 · COCO validation 1024×1024 | FID: 32.71 -> 30.86 | Helped |
| replace SD-v1.5 attention with initialized GSPN | SD-v1.5 · COCO validation 1024×1024 | CLIP-T: 0.290 -> 0.307 | Helped |
| remove pretrained SD query, key, and value initialization (ablation) | SD-v1.5-GSPN · COCO validation 1024×1024 | FID: 30.86 -> 36.89 | Hurt |
| remove pretrained SD query, key, and value initialization (ablation) | SD-v1.5-GSPN · COCO validation 1024×1024 | CLIP-T: 0.307 -> 0.278 | Hurt |
| replace all-local GSPN with a hybrid of local and global GSPN (ablation) | GSPN-T · ImageNet classification | Top-1 accuracy: 81.2% -> 0.4% accuracy improvement | Helped |
| replace SPN absolute normalization and max-pooling merge with sigmoid normalization and learnable merging (ablation) | GSPN-T · ImageNet classification | Top-1 accuracy: 81.2% -> 82.3% | Helped |
| remove positional embeddings and GLU, adding layers to maintain parameter count (ablation) | GSPN-T · ImageNet classification | Top-1 accuracy: +2% | Helped |
| increase hidden dimensions by 1.5× and add GRN layers (ablation) | GSPN-T · ImageNet classification | Top-1 accuracy: +1.0% | Helped |
| remove row normalization (ablation) | GSPN-T · ImageNet classification | training stability: about 40 epochs | Hurt |
Surprising findings
- Row normalization is essential: GSPN-T without it suffered gradient explosion (NaNs) after about 40 epochs.
- Removing positional embeddings and GLU did not hurt and in ablation contributed to accuracy gains, suggesting the scanning inherently encodes position.
Limitations
- The customized CUDA kernel runs slower than theoretical efficiency as channel and batch size grow due to inefficient memory access and no shared memory use.
- Speed advantage is minimal below 2K resolution; the largest gains appear only at high resolution.
- Text-to-image experiments rely on distillation from a teacher on a curated 169k-image subset.
Experimental setup
Datasets: ImageNet-1K LAION MS-COCO
Benchmarks: ImageNet-1K MS-COCO
Metrics: top-1 accuracy FID sFID Inception Score Precision Recall CLIP-T inference speed
Baselines: ConvNeXt Swin Transformer VMamba LocalVMamba Vim DiT-XL/2 SiT-XL/2 U-ViT-H/2 PixArt-α Mamba Mamba2 Linfusion
Builds on: Spatial Propagation Network Mamba linear attention DiT Stable Diffusion
Closest papers by meaning
- Scaling Parallel Sequence Models to Vision Foundation Models (CVPR 2026 · similarity 0.95)
- GSPN-2: Efficient Parallel Sequence Modeling (NeurIPS 2025 · similarity 0.93)
- ViG: Linear-complexity Visual Sequence Learning with Gated Linear Attention (AAAI 2025 · similarity 0.85)
- SPTNet: An Efficient Alternative Framework for Generalized Category Discovery with Spatial Prompt Tuning (ICLR 2024 · similarity 0.85)
- Spatial-Mamba: Effective Visual State Space Models via Structure-Aware State Fusion (ICLR 2025 · similarity 0.84)
- Recursive Generalization Transformer for Image Super-Resolution (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.