Research map / Detection and segmentation / CNN features
PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use perceptual codebook targets instead of BEiT targets | ViT-B masked-image pre-training · ImageNet-1K | Top-1 accuracy: 83.2 -> 84.5 | Helped |
| use perceptual codebook targets instead of BEiT targets | ViT-B masked-image pre-training · ADE20K | mIoU: 47.1 -> 48.5 | Helped |
| use perceptual codebook targets instead of MAE pixel targets | ViT-B masked-image pre-training · COCO val | box AP: 46.8 -> 47.8 | Helped |
| use perceptual codebook targets instead of MAE pixel targets | ViT-B masked-image pre-training · COCO val | mask AP: 41.9 -> 42.6 | Helped |
| use perceptual codebook targets instead of BEiT targets | TimeSformer initialized from ImageNet-1K pre-training · SSv2 | accuracy: 60.2 -> 61.8 | Helped |
| add perceptual loss to pixel loss during tokenizer training (ablation) | PeCo · ImageNet-1K | Top-1 accuracy: 82.9 -> 84.1 | Helped |
| use SSL ViT-B features rather than SSL ResNet-50 features for perceptual loss (ablation) | PeCo tokenizer · ImageNet-1K | Top-1 accuracy: 84.0 -> 84.1 | No clear effect |
| use supervised VGG rather than SSL ViT-B features for perceptual loss (ablation) | PeCo tokenizer · ImageNet-1K | Top-1 accuracy: 84.1 -> 84.1 | No clear effect |
| replace image-level perceptual loss with classification loss on codewords (ablation) | PeCo tokenizer · ImageNet-1K | Top-1 accuracy: 84.1 -> 82.9 | Hurt |
| replace image-level perceptual loss with contrastive loss on codewords (ablation) | PeCo tokenizer · ImageNet-1K | Top-1 accuracy: 84.1 -> 82.9 | Hurt |
| add adversarial loss to pixel and perceptual losses (ablation) | PeCo tokenizer · ImageNet-1K | Top-1 accuracy: 84.1 -> 83.9 | Hurt |
| increase perceptual-loss weight from 1 to 3 (ablation) | PeCo tokenizer · ImageNet-1K | Top-1 accuracy: 84.1 -> 83.6 | Hurt |
Closest papers by meaning
- BEiT: BERT Pre-Training of Image Transformers (ICLR 2022 · similarity 0.88)
- Training Object Detectors From Scratch: An Empirical Study in the Era of Vision Transformer (CVPR 2022 · similarity 0.87)
- Perception Encoder: The best visual embeddings are not at the output of the network (NeurIPS 2025 · similarity 0.86)
- Vision Transformers Need Registers (ICLR 2024 · similarity 0.86)
- The Unreasonable Effectiveness of Linear Prediction as a Perceptual Metric (ICLR 2024 · similarity 0.84)
- On the Surprising Effectiveness of Attention Transfer for Vision Transformers (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.