Research map / Architectures and efficiency / Attention and transformers
Transformer-Based Attention Networks for Continuous Pixel-Wise Prediction
Guanglei Yang, Hao Tang, Mingli Ding, Nicu Sebe, Elisa Ricci
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add attention gate decoder (AGD) (ablation) | ResNet-50 baseline · KITTI (Eigen split) | delta<1.25: 0.888 -> 0.953 | Improved |
| add ViT Transformer layers (ablation) | ResNet-50 baseline · KITTI (Eigen split) | delta<1.25: 0.888 -> 0.955 | Improved |
| add both AGD and ViT (ablation) | ResNet-50 baseline · KITTI (Eigen split) | delta<1.25: 0.888 -> 0.956 | Improved |
| add AGD (ablation) | ResNet-50 baseline · NYU Depth v2 | delta<1.25: 0.866 -> 0.881 | Improved |
| add ViT Transformer layers (ablation) | ResNet-50 baseline · NYU Depth v2 | delta<1.25: 0.866 -> 0.887 | Improved |
| add both AGD and ViT (ablation) | ResNet-50 baseline · NYU Depth v2 | delta<1.25: 0.866 -> 0.900 | Improved |
| add emitting feature f4 to f5 fusion (ablation) | AGD · NYU Depth v2 (fusion ablation) | delta<1.25: 0.878 -> 0.897 | Improved |
| add emitting feature f3 (three scales) (ablation) | AGD · NYU Depth v2 (fusion ablation) | delta<1.25: 0.897 -> 0.900 | Improved |
| add fourth emitting feature f2 (ablation) | AGD · NYU Depth v2 (fusion ablation) | rel: 0.106 -> 0.107 | Hurt |
| self-attention gating with only f5 as emitting and receiving feature (ablation) | AGD · NYU Depth v2 (fusion ablation) | delta<1.25: 0.866 -> 0.878 | Improved |
| use hybrid R50+ViT-B/16 instead of pure ViT-B/16 (ablation) | TransDepth encoder · NYU Depth v2 (backbone ablation) | delta<1.25: 0.885 -> 0.900 | Improved |
| use ViT-B/16 instead of ViT-B/32 on ResNet-50 (ablation) | hybrid encoder · NYU Depth v2 (backbone ablation) | delta<1.25: 0.893 -> 0.900 | Improved |
Closest papers by meaning
- Long-Short Transformer: Efficient Transformers for Language and Vision (NeurIPS 2021 · similarity 0.88)
- Transformer Meets Twicing: Harnessing Unattended Residual Information (ICLR 2025 · similarity 0.87)
- Viewing Transformers Through the Lens of Long Convolutions Layers (ICML 2024 · similarity 0.87)
- Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers (Student Abstract) (AAAI 2024 · similarity 0.86)
- AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers (ICML 2024 · similarity 0.86)
- CAT: Circular-Convolutional Attention for Sub-Quadratic Transformers (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
