Research map / Architectures and efficiency / Attention and transformers
ViG: Linear-complexity Visual Sequence Learning with Gated Linear Attention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ViG, a vision backbone built on bidirectional gated linear attention with direction-wise gating and 2D gating locality injection, plus a fused bidirectional kernel for faster wall-time at high resolution.
Problem
Linear-complexity sequence models (Mamba, RWKV, etc.) achieve ViT-like accuracy with fewer FLOPs but do not translate into real wall-time speedups at common resolutions, and unidirectional linear models lack global 2D context for vision.
Approach
Adapts gated linear attention (GLA) to vision via bidirectional scanning that shares parameters across directions except a direction-specific forget gate, adds a learned gate that fuses depthwise-conv local features with BiGLA global features, and fuses forward and backward scans into one GPU kernel.
Key results
- ViG-S reaches 81.7% top-1 on ImageNet-1K at 224 with 23M params and 3.5G FLOPs, matching DeiT-B (81.8%) with 3.7x fewer parameters.
- At 1024x1024, ViG-T uses 5.2x fewer FLOPs, saves 90% GPU memory, runs 4.8x faster and is 20.7% more accurate than DeiT-T.
- ViG-H-B reaches 84.2% top-1 on ImageNet-1K, and ViG-B gets 47.3 AP^b on COCO with 498.5G backbone FLOPs versus 893.3G for ViT-B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the large-kernel patch embedding with the improved convolutional patch embedding (ablation) | GLA · ImageNet-1K val, 224² | top-1 accuracy: 66.1 -> 73.8 | Helped |
| Add direct bidirectional modeling (ablation) | Improved patch embedding · ImageNet-1K val, 224² | top-1 accuracy: 73.8 -> 75.2 | Helped |
| Add absolute position embedding (ablation) | Direct bidirectional modeling · ImageNet-1K val, 224² | top-1 accuracy: 75.2 -> 75.4 | Helped |
| Add direction-wise gating (ablation) | Bidirectional GLA · ImageNet-1K val, 224² | top-1 accuracy: 75.4 -> 76.3 | Helped |
| Add 2D gating locality injection (ablation) | Direction-wise-gated bidirectional GLA · ImageNet-1K val, 224² | top-1 accuracy: 76.3 -> 77.2 | Helped |
| Fuse forward and backward scans into a hardware-aware implementation (ablation) | ViG-T · ImageNet-1K val, 224² | throughput: 3812 -> 4645 | Helped |
| Use direction-wise-gated bidirectional modeling rather than Vim-style extra backward GLA (ablation) | GLA · ImageNet-1K val, 224² | top-1 accuracy: 73.5 -> 77.2 | Helped |
| Use ViG-T instead of VRWKV-T | COCO val2017 · COCO val2017 | APᵇ: 41.7 -> 43.3 | Helped |
| Use ViG-B instead of ViT-B | COCO val2017 · COCO val2017 | APᵇ: 46.8 -> 47.3 | Helped |
| Use ViG-T instead of VRWKV-T | ADE20K val · ADE20K val | mIoU: 43.3 -> 43.8 | Helped |
Surprising findings
- Naively following Vim's bidirectional design on GLA (extra backward layer) yields 73.5% accuracy with lower throughput than ViG's shared-parameter design at 77.2%.
Limitations
- Still slightly slower than DeiT at 224x224 input resolution.
- Hardware-aware implementation still needs further optimization according to the authors.
Experimental setup
Datasets: ImageNet-1K COCO 2017 ADE20K
Benchmarks: ImageNet-1K COCO val2017 ADE20K
Metrics: top-1 accuracy FLOPs parameters throughput GPU memory latency box AP mask AP mIoU
Baselines: DeiT ViT Vim VRWKV VMamba LocalVim PlainMamba Swin Transformer ConvNeXt ResNet50
Builds on: gated linear attention Vision Mamba DeiT ViT-Adapter VRWKV
Closest papers by meaning
- Gated Linear Attention Transformers with Hardware-Efficient Training (ICML 2024 · similarity 0.87)
- Gated Slot Attention for Efficient Linear-Time Sequence Modeling (NeurIPS 2024 · similarity 0.86)
- DiG: Scalable and Efficient Diffusion Models with Gated Linear Attention (CVPR 2025 · similarity 0.86)
- Parallel Sequence Modeling via Generalized Spatial Propagation Network (CVPR 2025 · similarity 0.85)
- Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences (ICML 2024 · similarity 0.85)
- You Only Need Less Attention at Each Stage in Vision Transformers (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.