Research map / Architectures and efficiency / Attention and transformers
RIFormer: Keep Your Vision Backbone Effective but Removing Token Mixer
Jiahao Wang, Songyang Zhang, Yong Liu, Taiqiang Wu, Yujiu Yang, Xihui Liu, Kai Chen, Ping Luo, Dahua Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Remove the pooling token mixer and train with CE loss (ablation) | MetaFormer-S12 · ImageNet-1K | Top-1 accuracy (%): 75.01 -> 72.31 | Hurt |
| Use soft teacher distillation without ground-truth labels (ablation) | RIFormer-S12 with identity mapping · ImageNet-1K | Top-1 accuracy (%): 72.31 -> 74.05 | Improved |
| Use affine transformation without tailored distillation (ablation) | RIFormer-S12 · ImageNet-1K | Top-1 accuracy (%): 72.31 -> 72.25 | No clear effect |
| Replace identity mapping with affine transformation while using soft distillation without labels (ablation) | RIFormer-S12 · ImageNet-1K | Top-1 accuracy (%): 74.05 -> 74.07 | No clear effect |
| Add module imitation to the affine student (ablation) | RIFormer-S12 · ImageNet-1K | Top-1 accuracy (%): 74.07 -> 75.13 | Improved |
| Use a teacher with a large receptive field, GFNet-H-B, rather than other tested teachers (ablation) | RIFormer-S12 · ImageNet-1K | Top-1 accuracy (%): 73.63; 73.12; 73.25; 74.07 -> 74.83; 74.52; 74.25; 75.13 | Mixed |
| Initialize the student weights from the teacher, excluding the affine operator (ablation) | RIFormer-S12 · ImageNet-1K | Top-1 accuracy (%): 75.13% to 75.36% | Improved |
| Replace module imitation with hidden-state feature distillation (ablation) | RIFormer-S12 · ImageNet-1K | Top-1 accuracy (%): 75.36 -> 74.90 | Hurt |
| Remove the teacher from training (ablation) | RIFormer-S12 · ImageNet-1K | Top-1 accuracy (%): 72.75 -> 75.62 | Hurt |
| Use RIFormer token-mixer removal instead of depth or width slimming (ablation) | PoolFormer-S12 · ImageNet-1K | Top-1 accuracy (%): PoolFormer-S9: 74.78; PoolFormer-XS12: 75.11; RIFormer-S12: 75.36 | Mixed |
| Remove token mixers from the RIFormer building blocks | RIFormer-M36 · ImageNet-1K val, 224 | Top-1 accuracy (%): 82.1 -> 82.6 | Improved |
| Remove token mixers from the RIFormer building blocks | RIFormer-S12 · ImageNet-1K val, 224 | Throughput (images/s): 4160.18 -> 4899.80 | Improved |
Closest papers by meaning
- MetaFormer Is Actually What You Need for Vision (CVPR 2022 · similarity 0.88)
- HydraViT: Stacking Heads for a Scalable ViT (NeurIPS 2024 · similarity 0.86)
- Vision Transformers with Self-Distilled Registers (NeurIPS 2025 · similarity 0.85)
- Frequency-Aware Token Reduction for Efficient Vision Transformer (NeurIPS 2025 · similarity 0.84)
- FreeNet: Liberating Depth-Wise Separable Operations for Building Faster Mobile Vision Architectures (AAAI 2025 · similarity 0.84)
- TransNeXt: Robust Foveal Visual Perception for Vision Transformers (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
