AI Paper Atlas

Research map / Detection and segmentation / Weakly supervised detection and segmentation

Vision Transformers Are Good Mask Auto-Labelers

Shiyi Lan, Xitong Yang, Zhiding Yu, Zuxuan Wu, Jose M. Alvarez, Anima Anandkumar

CVPR 2023 · Detection and segmentation › Weakly supervised detection and segmentation · approach: Semi-supervised instance segmentation · 21 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use attention-based mask decoder instead of fully connected or fully convolutional decoder in phase 1 (ablation)MAL mask decoder · COCO val2017Mask AP: 42.3 vs 35.5 (FC) and 36.1 (FCN)Improved
use Mask2Former-style query-based decoder in phase 1 (ablation)MAL mask decoder · COCO val2017Mask AP: no reasonable results obtainedHurt
use MAE pretraining for ViT encoder instead of DeiT or Swin/ConvNeXt encoders (ablation)MAL image encoder · COCO val2017Mask AP: 42.3 (MAE-Base) vs 41.1 (DeiT-Base), 40.8 (DeiT-Small), 40.2 (Swin-Base), 39.6 (ConvNeXt-Base)Improved
scale image encoder from ViT-MAE-Base to ViT-MAE-Large (ablation)MAL image encoder · COCO val2017Mask AP: 42.3 for both Base and LargeNo clear effect
box expansion upper bound theta increased from 0.6 to 1.2 (ablation)MAL RoI input sampling · COCO val2017Mask AP: 41.3 at theta 0.6 to 42.3 at theta 1.2Improved
increase box expansion upper bound theta beyond 1.2 to 1.6 (ablation)MAL RoI input sampling · COCO val2017Mask AP: 42.0 at 1.4 and 41.8 at 1.6 vs 42.3 at 1.2Hurt
use box expansion RoI sampling with background bags rather than pure box cropping (ablation)MAL RoI input generation · COCOqualitative mask qualityImproved
use MAL-generated masks to train Mask2Former (Swin-Small) instead of GT maskstwo-phase box-supervised pipeline · COCO test-dev 2017Mask AP: 44.1 test-dev vs 40.0 for BoxTeacher box-supervisedImproved
train MAL on COCO and generate LVIS pseudo-labels (open-vocabulary) instead of training on LVISMAL labeler · LVIS v1 valMask AP: about 0.35% mAP drop vs LVIS-trained (23.3 vs 23.7 with X101-32x4d, 24.2 vs 24.5 with X101-64x4d)No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy