AI Paper Atlas

Research map / Detection and segmentation / CNN features

Mc-BEiT: Multi-Choice Discretization for Image BERT Pre-training

Xiaotong Li, Yixiao Ge, Kun Yi, Zixuan Hu, Ying Shan, Ling-Yu Duan

ECCV 2022 · Detection and segmentation › CNN features · approach: Convolution-transformer backbones · 38 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add multi-choice soft targets and patch-similarity refinement (mc-BEiT vs BEiT)BEiT ViT-B 800 ep · ImageNet-1K fine-tuningtop-1 accuracy: 83.2 -> 84.1Improved
add multi-choice objective (mc-BEiT vs BEiT)BEiT ViT-L 800 ep · ImageNet-1K fine-tuningtop-1 accuracy: 85.2 -> 85.6Improved
use DALL-E tokenizer with multi-choice objective instead of single-choice (ablation)BEiT, 100 epochs · ImageNet-1Ktop-1 accuracy: 82.3 -> 82.6Improved
use VQGAN tokenizer with multi-choice objective instead of single-choice (ablation)BEiT, 100 epochs · ImageNet-1Ktop-1 accuracy: 82.9 -> 83.3Improved
use VQGAN tokenizer instead of DALL-E (ablation)BEiT, 100 epochs · ImageNet-1Ktop-1 accuracy: 82.3 -> 82.9Improved
use VQGAN tokenizer instead of DALL-E (ablation)mc-BEiT, 800 epochs · ImageNet-1Ktop-1 accuracy: 83.7 -> 84.1Improved
add tokenizer-prediction target (omega 0 -> 0.8) rather than only patch-similarity target (ablation)mc-BEiT, 100 epochs · ImageNet-1Ktop-1 accuracy: 81.8 -> 83.3Improved
use random 75% masking instead of block 75% (ablation)mc-BEiT, 100 epochs · ImageNet-1Ktop-1 accuracy: 82.8 -> 83.3Improved
raise random mask ratio from 75% to 90% (ablation)mc-BEiT, 100 epochs · ImageNet-1Ktop-1 accuracy: 83.3 -> 83.0Hurt
lower random mask ratio from 75% to 45% (ablation)mc-BEiT, 100 epochs · ImageNet-1Ktop-1 accuracy: 83.3 -> 83.0Hurt
increase pre-training from 100 to 800 epochs (ablation)mc-BEiT ViT-B · ImageNet-1Ktop-1 accuracy: 83.3 -> 84.1Improved
add intermediate ImageNet fine-tuning before ADE20K (ablation)mc-BEiT ViT-B · ADE20KmIoU: 47.0 -> 50.8Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy