Research map / Architectures and efficiency / Attention and transformers
Boosting Vanilla Lightweight Vision Transformers via Re-parameterization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes re-parameterization for vanilla lightweight Vision Transformers to close the gap with lightweight CNNs and hybrid networks.
Problem
Vanilla lightweight Vision Transformers still perform far worse than recent lightweight CNNs or hybrid networks.
Approach
Adapts the widely used re-parameterization technique from CNNs to vanilla lightweight ViTs to unlock their potential without changing the inference-time architecture.
Experimental setup
Builds on: structural re-parameterization
Closest papers by meaning
- Slicing Vision Transformer for Flexible Inference (NeurIPS 2024 · similarity 0.88)
- RepViT: Revisiting Mobile CNN From ViT Perspective (CVPR 2024 · similarity 0.87)
- SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization (ICML 2024 · similarity 0.87)
- RMT: Retentive Networks Meet Vision Transformers (CVPR 2024 · similarity 0.87)
- Memory Efficient Transformer Adapter for Dense Predictions (ICLR 2025 · similarity 0.87)
- Structured Initialization for Vision Transformers (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.