Research map / Detection and segmentation / CNN features
Masked Image Modeling with Denoising Contrast
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use denoising patch-level contrast rather than vanilla instance-level contrast (ablation) | ConMIM · ImageNet-1K, ViT-B/16, 300 pre-training epochs | top-1 accuracy: 82.26 -> 83.51; -1.25% when replaced | Helped |
| Remove masking while keeping patch-level contrast (ablation) | ConMIM · ImageNet-1K, ViT-B/16, 300 pre-training epochs | top-1 accuracy: fail | Hurt |
| Keep negative keys intra-image rather than expanding them to intra-GPU (ablation) | ConMIM · ImageNet-1K, ViT-B/16, 300 pre-training epochs | top-1 accuracy: 82.60 -> 83.51; -0.91% when expanded | Helped |
| Retain highly similar intra-image negative keys instead of filtering them (ablation) | ConMIM · ImageNet-1K, ViT-B/16, 300 pre-training epochs | top-1 accuracy: 83.12 -> 83.51; -0.39% when filtered | Helped |
| Use stronger augmentation on the full input and basic augmentation on the corrupted input (ablation) | ConMIM · ImageNet-1K, ViT-B/16, 300 pre-training epochs | top-1 accuracy: 83.41 -> 83.51; 83.35 -> 83.51 | Helped |
| Use stronger augmentation on the corrupted input and basic augmentation on the full input (ablation) | ConMIM · ImageNet-1K, ViT-B/16, 300 pre-training epochs | top-1 accuracy: 83.51 -> 82.62; -0.89% | Hurt |
| Use a momentum encoder for full-input target representations (ablation) | ConMIM · ImageNet-1K, ViT-B/16, 300 pre-training epochs | top-1 accuracy: 81.53 -> 83.51; -1.98% when removed | Helped |
| Use random 75% masking rather than other tested strategies or ratios (ablation) | ConMIM · ImageNet-1K, ViT-B/16 | top-1 accuracy: Block: 40% 82.12; 60% 83.20; 75% 83.11; 90% 82.63. Random: 60% 82.52; 75% 83.51; 90% 83.04 | Mixed |
| Use temperature 0.1 rather than the tested alternatives (ablation) | ConMIM · ImageNet-1K, ViT-B/16 | top-1 accuracy: 0.05 82.55; 0.07 83.03; 0.1 83.51; 0.15 83.32; 0.2 83.35 | Mixed |
| Set momentum coefficient to 0.996 rather than 0.99 or 0.999 (ablation) | ConMIM · ImageNet-1K, ViT-B/16 | top-1 accuracy: 0.99 83.11; 0.996 83.51; 0.999 83.31 | No clear effect |
| Pre-train with ConMIM rather than BEiT | ViT-S/16 · ImageNet-1K classification, 300 epochs | top-1 accuracy: 81.3 -> 82.0 | Helped |
| Pre-train with ConMIM rather than BEiT | ViT-B/16 · ADE20K semantic segmentation, 800 epochs, intermediate fine-tuning | mIOU: 47.7 -> 49.8 | Helped |
Closest papers by meaning
- Contrastive Tuning: A Little Help to Make Masked Autoencoders Forget (AAAI 2024 · similarity 0.89)
- Hybrid Distillation: Connecting Masked Autoencoders with Contrastive Learners (ICLR 2024 · similarity 0.89)
- On the Role of Discrete Tokenization in Visual Representation Learning (ICLR 2024 · similarity 0.89)
- From Prototypes to General Distributions: An Efficient Curriculum for Masked Image Modeling (CVPR 2025 · similarity 0.88)
- Wavelet-Driven Masked Image Modeling: A Path to Efficient Visual Representation (AAAI 2025 · similarity 0.88)
- Multi-Modal Contrastive Masked Autoencoders: A Two-Stage Progressive Pre-training Approach for RGBD Datasets (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.