Research map / Detection and segmentation / CNN features
Could Giant Pre-trained Image Models Extract Universal Representations?
Yutong Lin, Ze Liu, Zheng Zhang, Han Hu, Nanning Zheng, Stephen Lin, Yue Cao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace supervised ImageNet-1K pretraining with SimMIM masked image modeling (ablation) | frozen Swin · COCO | AP box: 42.4 -> 34.1 | Hurt |
| Pretrain SimMIM then fully finetune instead of freezing (ablation) | SimMIM-1K · COCO | AP box: 34.1 -> 52.9 | Improved |
| Scale supervised pretraining data from ImageNet-1K to ImageNet-22K (ablation) | frozen Swin · Kinetics-400 | top-1 accuracy: 60.4 -> 70.3 | Improved |
| Replace supervised with contrastive EsViT pretraining on ImageNet-1K (ablation) | frozen Swin · COCO | AP box: 42.4 -> 42.0 | No clear effect |
| Replace FPN with BiFPN in the head (ablation) | frozen SwinV2-B-22K · COCO | AP box: 45.0 -> 51.9 | Improved |
| Add four residual blocks inside FPN (ablation) | frozen SwinV2-B-22K · COCO | AP box: 45.0 -> 49.5 | Improved |
| Replace the box/mask head with a cascade head (ablation) | frozen SwinV2-B-22K · COCO | AP box: 45.0 -> 49.0 | Improved |
| Enlarge pixel decoder from 1 to 6 blocks (ablation) | frozen Mask2Former · ADE20K | mIoU: 51.0 -> 52.6 | Improved |
| Enlarge transformer decoder from 4 to 10 blocks (ablation) | frozen Mask2Former · ADE20K | mIoU: 51.0 -> 51.9 | Improved |
| Replace linear head with four temporal Transformer blocks (ablation) | frozen SwinV2-B-22K · Kinetics-400 | top-1 accuracy: 70.3 -> 74.7 | Improved |
| Use four global spatiotemporal blocks instead of temporal blocks (ablation) | frozen SwinV2-B-22K · Kinetics-400 | top-1 accuracy: 74.7 -> 77.5 | Improved |
| Add 3 BiFPN layers to head under 1% COCO data (frozen vs full finetune) (ablation) | frozen vs full ft. SwinV2-B-22K · COCO 1% | AP box: 32.1 vs 25.9 | Mixed |
Closest papers by meaning
- The Effectiveness of MAE Pre-Pretraining for Billion-Scale Pretraining (ICCV 2023 · similarity 0.88)
- Rethinking ImageNet Pre-Training (ICCV 2019 · similarity 0.88)
- Frozen-DETR: Enhancing DETR with Image Understanding from Frozen Foundation Models (NeurIPS 2024 · similarity 0.87)
- Frozen Transformers in Language Models Are Effective Visual Encoder Layers (ICLR 2024 · similarity 0.86)
- Scalable Pre-training of Large Autoregressive Image Models (ICML 2024 · similarity 0.86)
- What Would Gauss Say About Representations? Probing Pretrained Image Models using Synthetic Gaussian Benchmarks (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
