Research map / Label-efficient and robust learning / Self-supervised and contrastive
ResMLP: Feedforward Networks for Image Classification With Data-Efficient Training
Hugo Touvron, Piotr Bojanowski, Mathilde Caron, Matthieu Cord, Alaaeldin El-Nouby, Édouard Grave, Gautier Izacard, Armand Joulin, Gabriel Synnaeve, Jakob Jozef Verbeek, Hervé Jeǵou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LayerNorm instead of the Aff affine transform (ablation) | ResMLP-S12 · ImageNet-1k val | top-1 accuracy: 76.6 -> 77.7 | Improved |
| use Class-MLP pooling instead of average pooling (ablation) | ResMLP-S12 · ImageNet-1k val | top-1 accuracy: 76.6 -> 77.5 | Improved |
| add the cross-patch linear layer (ablation) | ResMLP-S12 · ImageNet-1k val | top-1 accuracy: 56.5 -> 76.6 | Improved |
| use 8x8 patches instead of 16x16 (ablation) | ResMLP-S12 · ImageNet-1k val | top-1 accuracy: 76.6 -> 79.1 | Improved |
| add distillation from a convnet (ablation) | ResMLP-S12 · ImageNet-1k val | top-1 accuracy: 76.6 -> 77.8 | Improved |
| add distillation from a convnet (ablation) | ResMLP-B24/8 · ImageNet-1k val | top-1 accuracy: 81.0 -> 83.6 | Improved |
| use the DeiT training recipe instead of old ResNet-style 90-epoch training (ablation) | ResMLP-S12 · ImageNet-1k val | top-1 accuracy: 69.2 -> 76.6 | Improved |
| pre-train on ImageNet-21k (ablation) | ResMLP-B24/8 · ImageNet-1k val | top-1 accuracy: 81.0 -> 84.4 | Improved |
| add RandAugment (ablation) | ResMLP-S12 · ImageNet-1k val | top-1 accuracy: 73.2 -> 76.6 | Improved |
| add DINO self-supervised pre-training before supervised fine-tuning (ablation) | ResMLP-S24 · ImageNet-1k val | top-1 accuracy: 79.4 -> 79.9 | Improved |
| train for 800 epochs instead of 300 (ablation) | ResMLP-S36 · ImageNet-1k val | top-1 accuracy: 79.3 -> 80.4 | Improved |
| use SiLU instead of GELU (ablation) | ResMLP (CaiT-derived MLP baseline) · ImageNet-1k val | top-1 accuracy: 79.2 -> 78.7 | Hurt |
Closest papers by meaning
- Deep Residual Learning for Image Recognition (CVPR 2016 · similarity 0.88)
- Big Self-Supervised Models are Strong Semi-Supervised Learners (NeurIPS 2020 · similarity 0.88)
- Multi-Scale Dense Networks for Resource Efficient Image Classification (ICLR 2018 · similarity 0.87)
- Revisiting ResNets: Improved Training and Scaling Strategies (NeurIPS 2021 · similarity 0.87)
- Residual Alignment: Uncovering the Mechanisms of Residual Networks (NeurIPS 2023 · similarity 0.87)
- No Pains, More Gains: Recycling Sub-Salient Patches for Efficient High-Resolution Image Recognition (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
