Research map / Detection and segmentation / CNN features

MIM-Refiner: A Contrastive Learning Boost from Intermediate Pre-Trained Masked Image Modeling Representations

ICLR 2025 · Detection and segmentation › CNN features · approach: Convolution-transformer backbones

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MIM-Refiner, a contrastive learning boost applied to pre-trained masked image modeling models by attaching nearest-neighbor instance discrimination heads to intermediate layers.

Problem

Pre-trained MIM models produce strong representations in intermediate layers rather than the final layer, and their final-layer features are weak for linear probing; the paper targets improving MIM representations.

Approach

Attach multiple instance discrimination heads to different intermediate layers of a pre-trained MIM model, each using a nearest-neighbor ID objective that constructs positive pairs from nearest neighbors, to refine representations with contrastive learning.

Surprising findings

• Strong MIM representations reside in intermediate layers rather than the final layer.

Experimental setup

Builds on: masked image modeling instance discrimination

Closest papers by meaning

• Contrastive Tuning: A Little Help to Make Masked Autoencoders Forget (AAAI 2024 · similarity 0.90)
• Hybrid Distillation: Connecting Masked Autoencoders with Contrastive Learners (ICLR 2024 · similarity 0.89)
• From Prototypes to General Distributions: An Efficient Curriculum for Masked Image Modeling (CVPR 2025 · similarity 0.88)
• On the Role of Discrete Tokenization in Visual Representation Learning (ICLR 2024 · similarity 0.86)
• Morphing Tokens Draw Strong Masked Image Models (ICLR 2025 · similarity 0.86)
• Efficient Vision-Language Pre-training by Cluster Masking (CVPR 2024 · similarity 0.86)