Research map / Detection and segmentation / CNN features
Unleashing Vanilla Vision Transformer with Masked Image Modeling for Object Detection
Yuxin Fang, Shusheng Yang, Shijie Wang, Yixiao Ge, Ying Shan, Xinggang Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| randomly sample 50% of input embeddings for ViT encoder during training instead of 100% (ablation) | MimDet-Base · COCO val2017 | APbox: 51.8 -> 51.5 | No clear effect |
| reduce training sampling ratio from 50% to 25% (ablation) | MimDet-Base · COCO val2017 | APbox: 51.5 -> 49.9 | Hurt |
| use random sampling instead of grid sampling (ablation) | MimDet-Base · COCO val2017 | APbox: 48.7 -> 51.5 | Improved |
| add a short full-input fine-tuning stage after sampled fine-tuning (ablation) | MimDet-Base · COCO val2017 | APbox: 51.5 -> 51.4 | No clear effect |
| infer with full input instead of ensembling sampled inferences (ablation) | MimDet-Base · COCO val2017 | APbox: 51.0 (8x50%) -> 51.5 (full) | Improved |
| replace naive ConvStem with ConvNeXt early stage (pretrained or not) (ablation) | MimDet-Base · COCO val2017 | APbox: 51.5 -> 51.3 / 51.4 | No clear effect |
| fill unsampled positions with ConvStem features instead of [MASK] tokens (ablation) | MimDet-Base · COCO val2017 | APbox: 51.5 -> 51.7 | Improved |
| increase decoder layers from 1 to 4 (ablation) | MimDet-Base · COCO val2017 | APbox: 49.9 -> 51.5 | Improved |
| increase decoder layers from 4 to 8 (ablation) | MimDet-Base · COCO val2017 | APbox: 51.5 -> 51.6 | No clear effect |
| take FPN high-resolution inputs from ConvStem instead of ViT intermediate features (ablation) | MimDet-Base · COCO val2017 | APbox: 48.9 -> 49.5 | Improved |
| use random sampled partial input instead of window partitioned input (ablation) | MimDet-Base (1 decoder layer) · COCO val2017 | APbox: 49.5 -> 49.9 | Improved |
| use MimDet instead of prior adapted vanilla ViT detector | MAE ViT-Base · COCO val2017 | APbox: 50.3 -> 51.7 | Improved |
Closest papers by meaning
- A Simple Single-Scale Vision Transformer for Object Detection and Instance Segmentation (ECCV 2022 · similarity 0.90)
- Exploring Plain Vision Transformer Backbones for Object Detection (ECCV 2022 · similarity 0.90)
- Efficient Decoder-Free Object Detection with Transformers (ECCV 2022 · similarity 0.90)
- Integrally Migrating Pre-trained Transformer Encoder-decoders for Visual Object Detection (ICCV 2023 · similarity 0.90)
- MCMAE: Masked Convolution Meets Masked Autoencoders (NeurIPS 2022 · similarity 0.89)
- Self-Supervised Learning of Intertwined Content and Positional Features for Object Detection (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
