Research map / Detection and segmentation / Open-set and few-shot detection
Contrastive Feature Masking Open-Vocabulary Vision Transformer
Dahun Kim, Anelia Angelova, Weicheng Kuo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add masked feature reconstruction in joint image-text embedding space (ablation) | contrastive image-text pretraining · LVIS OVD | APr: 27.4 -> 30.7 | Improved |
| reconstruct raw pixels or first-layer features instead of final features (ablation) | CFM-ViT · LVIS OVD | APr: 27.4 -> 27.1 (pixels), 27.2 (1st-layer) | No clear effect |
| add positional embedding dropout in pretraining (ablation) | contrastive baseline · LVIS OVD | APr: 27.4 -> 28.5 | Improved |
| PED plus masked feature reconstruction (ablation) | CFM-ViT · LVIS OVD | APr: 27.4 -> 31.2 | Improved |
| remove positional embeddings entirely instead of dropout (ablation) | ViT encoder · LVIS OVD | APr: 27.4 -> 25.8 | Hurt |
| mask contrastive branch to 75% tokens, disjoint from reconstruction input (ablation) | CFM-ViT · LVIS OVD | APr and FLOPs: FLOPs 1.23x -> 1.01x; APr 30.7 -> 30.4 | Improved |
| use backbone lr ratio 0.5 instead of 0.0, 0.1 or 1.0 (ablation) | detection finetuning · LVIS OVD | APr: 26.0 -> 27.4 | Improved |
| frozen backbone inference for region classification, with PED pretraining (ablation) | CFM-ViT · LVIS OVD | APr: 31.2 -> 32.5 | Improved |
| frozen backbone inference without PED pretraining (ablation) | CFM-ViT · LVIS OVD | APr: 30.7 -> 27.1 | Hurt |
| swap pretraining data from ALIGN to LAION (ablation) | CFM-ViT · LVIS OVD | APr: 32.5 -> 32.4 (LAION-2B), 32.2 (LAION-400M) | No clear effect |
| add feature reconstruction plus PED (ablation) | contrastive baseline · Flickr30K zero-shot retrieval | image-to-text R@1: 86.0 -> 87.0 | Improved |
| increase batch size from 4k to 16k (ablation) | CFM-ViT · LVIS OVD | APr: ViT-B 26.8 -> 28.8; ViT-L 32.5 -> 33.9 | Improved |
Closest papers by meaning
- LLMs Meet VLMs: Boost Open Vocabulary Object Detection with Fine-grained Descriptors (ICLR 2024 · similarity 0.87)
- Contrastive Tuning: A Little Help to Make Masked Autoencoders Forget (AAAI 2024 · similarity 0.86)
- Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection (ICLR 2025 · similarity 0.85)
- Efficient Vision-Language Pre-training by Cluster Masking (CVPR 2024 · similarity 0.85)
- GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection (NeurIPS 2025 · similarity 0.85)
- Self-Supervised Learning of Intertwined Content and Positional Features for Object Detection (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
