Research map / Detection and segmentation / CNN features
TransNeXt: Robust Foveal Visual Perception for Vision Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TransNeXt, a hierarchical vision transformer built from Aggregated Attention (a per-pixel foveal token mixer combining sliding-window and pooled attention with learnable query embeddings and positional attention) and Convolutional GLU channel mixer, achieving strong ImageNet, robustness, detection and segmentation results.
Problem
Efficient vision transformers that rely on stacked layers for information exchange suffer from depth degradation, leading to insufficient information mixing and unnatural window-partition artifacts in visual perception.
Approach
A biomimetic per-pixel token mixer (Pixel-focused Attention) that pairs query-centered sliding-window attention (fine-grained foveal) with pooled global attention in a single softmax, extended by learnable query embedding and learnable positional tokens (Aggregated Attention), plus a Convolutional GLU channel mixer whose 3x3 depthwise conv gates channels based on nearest-neighbor features.
Key results
- TransNeXt-Tiny reaches 84.0% ImageNet-1K top-1 at 224 with 28.2M params, surpassing ConvNeXt-B with 69% fewer parameters.
- TransNeXt-Base reaches 61.6% ImageNet-A top-1 at 384, outperforming ConvNeXt-L by 10.9 points.
- TransNeXt-Base achieves 54.7 ADE20K mIoU with UperNet and COCO DINO-based box AP of 57.1 (5-scale).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TransNeXt-Tiny instead of MaxViT-Tiny | ImageNet-1K classification · ImageNet-1K val, 224² | top-1 accuracy: 83.4 -> 84.0 | Helped |
| Fine-tune TransNeXt-Base at 384² | TransNeXt-Base · ImageNet-1K val, 384² fine-tuned | top-1 accuracy: 85.7 -> 86.2 | Helped |
| Use TransNeXt-Base instead of ConvNeXt-L | ImageNet-A · ImageNet-A, 384² | top-1 accuracy: 50.7 -> 61.6 | Helped |
| Use TransNeXt-Tiny instead of ConvNeXt-L | COCO object detection with DINO · COCO, DINO 4-scales | APb: 53.4 -> 55.1 | Helped |
| Replace ConvFFN with GLU (ablation) | PVTv2-B1-derived model · ImageNet-1K val, ablation roadmap | top-1 accuracy: 80.12 -> 79.7 (-0.42) | Hurt |
| Replace GLU with ConvGLU (ablation) | PVTv2-B1-derived model · ImageNet-1K val, ablation roadmap | top-1 accuracy: 79.7 -> 80.9 (+1.2) | Helped |
| Replace SRA with PFA (ablation) | PVTv2-B1-derived model · ImageNet-1K val, ablation roadmap | top-1 accuracy: 80.9 -> 81.8 (+0.9) | Helped |
| Add positional attention to PFA (ablation) | PVTv2-B1-derived model · ImageNet-1K val, ablation roadmap | top-1 accuracy: 81.8 -> 82.2 (+0.4) | Helped |
| Add query embedding to positional-attention PFA (ablation) | PVTv2-B1-derived model · ImageNet-1K val, ablation roadmap | top-1 accuracy: 82.2 -> 82.5 (+0.3) | Helped |
| Replace ConvFFN with GLU (ablation) | PVTv2-B1-derived model · ImageNet-A, ablation roadmap | top-1 accuracy: 19.2 -> 18.9 (-0.3) | Hurt |
| Replace GLU with ConvGLU (ablation) | PVTv2-B1-derived model · ImageNet-A, ablation roadmap | top-1 accuracy: 18.9 -> 23.5 (+4.6) | Helped |
| Replace SRA with PFA (ablation) | PVTv2-B1-derived model · ImageNet-A, ablation roadmap | top-1 accuracy: 23.5 -> 26.9 (+3.4) | Helped |
Surprising findings
- Large-kernel convolutional models such as RepLKNet-31B fall to 0.9% accuracy at 640 resolution, while TransNeXt keeps its accuracy under multi-scale inference, challenging the assumption that pure CNNs generalize better across scales.
Limitations
- Stage 4 cannot use pooling-based attention at high resolution, so it falls back to modified MHSA
- Gains in dense tasks and ablations are reported on specific configurations; the 3x3 sliding window choice is empirically motivated
Experimental setup
Datasets: ImageNet-1K COCO ADE20K CIFAR-100
Benchmarks: ImageNet-1K ImageNet-A ImageNet-C ImageNet-R ImageNet-Sketch ImageNet-V2 COCO ADE20K
Metrics: top-1 accuracy mCE box AP mask AP mIoU
Baselines: PVTv2 ConvNeXt MaxViT FocalNet BiFormer Swin Transformer CSWin InternImage
Builds on: PVTv2 Swin Transformer ConvNeXt Gated Linear Unit Squeeze-and-Excitation
Closest papers by meaning
- Seeing More with Less: Human-like Representations in Vision Models (CVPR 2025 · similarity 0.86)
- Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials (NeurIPS 2025 · similarity 0.85)
- Perception Encoder: The best visual embeddings are not at the output of the network (NeurIPS 2025 · similarity 0.85)
- Causal Tracing of Object Representations in Large Vision Language Models: Mechanistic Interpretability and Hallucination Mitigation (AAAI 2026 · similarity 0.85)
- PerceptionGPT: Effectively Fusing Visual Perception into LLM (CVPR 2024 · similarity 0.85)
- CanViT: Toward Active-Vision Foundation Models (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.