Research map / Label-efficient and robust learning / Self-supervised and contrastive
Mixed Autoencoder for Self-Supervised Visual Representation Learning
Kai Chen, Zhili Liu, Lanqing Hong, Hang Xu, Zhenguo Li, Dit-Yan Yeung
This paper tried a common change: Add a contrastive loss
Add homologous contrastive loss alongside homologous attention · ADE20K, 300-epoch ablation · mIoU: 44.5 -> 46.4 Improved
What 1,317 accepted papers report for this change
- 1,253 improved
- 22 no clear effect
- 16 mixed
- 26 hurt
- Beyond Looking Up, Try Looking Around: Harmonizing Global Structure and Local Consistency in Optimal Transport for Short Text Clustering ICML 2026 · add contrastive learning · SearchSnippets · ACC: 56.65 -> 80.12Improved
- SleepLM: Natural-Language Intelligence for Human Sleep ICML 2026 · add contrastive objective to captioning (CoCa vs Cap) · sleep event IoU · IoU: 29.2 -> 27.4Hurt
- LIMSSR: LLM-Driven Sequence-to-Score Reasoning under Training-Time Incomplete Multimodal Observations ICML 2026 · add contrastive loss L_con · FS1000 average · Spearman correlation: 0.759 -> 0.789Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add homologous recognition to MAE-style mixing | MAE with ViT-Base · ImageNet-1K, 300 epochs | Top-1 accuracy: 82.7 -> 83.1 | Improved |
| Add homologous recognition to MAE-style mixing | MAE with ViT-Base · ADE20K, 300 epochs | mIoU: 46.1 -> 47.0 | Improved |
| Add homologous recognition to MAE-style mixing | MAE with ViT-Base · COCO, 300 epochs | box AP: 47.2 -> 47.8 | Improved |
| Add homologous recognition to MAE-style mixing | MAE with ViT-Base · ImageNet-1K, 1600 epochs | Top-1 accuracy: 83.6 -> 83.9 | Improved |
| Set the mixing ratio to r=0.25 rather than r=0.5 (ablation) | MixedAE · ImageNet-1K, 300-epoch ablation | Top-1 accuracy: 82.3 -> 82.4 | Improved |
| Set the mixing ratio to r=0.25 rather than r=0.5 (ablation) | MixedAE · ADE20K, 300-epoch ablation | mIoU: 44.2 -> 45.0 | Improved |
| Apply homologous contrastive loss after LayerNorm rather than before it (ablation) | MixedAE · ADE20K, 300-epoch ablation | mIoU: 44.5 -> 45.3 | Improved |
| Use each homologous positive separately instead of averaging positives (ablation) | MixedAE · ADE20K, 300-epoch ablation | mIoU: 45.1 -> 45.3 | Improved |
| Set homologous-attention threshold K to 0.25L instead of disabling TopK sampling (ablation) | MixedAE · ADE20K, 300-epoch ablation | mIoU: 45.3 -> 45.6 | Improved |
| Keep global self-attention in the first layer only (ablation) | MixedAE · ADE20K, 300-epoch ablation | mIoU: 45.6 -> 46.4 | Improved |
| Add homologous contrastive loss alongside homologous attention (ablation) | MixedAE · ADE20K, 300-epoch ablation | mIoU: 44.5 -> 46.4 | Improved |
| Add homologous attention alongside homologous contrastive loss (ablation) | MixedAE · ADE20K, 300-epoch ablation | mIoU: 45.3 -> 46.4 | Improved |
Closest papers by meaning
- Contrastive Tuning: A Little Help to Make Masked Autoencoders Forget (AAAI 2024 · similarity 0.90)
- Self-Guided Masked Autoencoder (NeurIPS 2024 · similarity 0.88)
- Learning Mask Invariant Mutual Information for Masked Image Modeling (ICLR 2025 · similarity 0.88)
- The Dynamic Duo of Collaborative Masking and Target for Advanced Masked Autoencoder Learning (AAAI 2025 · similarity 0.87)
- Hybrid Distillation: Connecting Masked Autoencoders with Contrastive Learners (ICLR 2024 · similarity 0.87)
- Exploring Target Representations for Masked Autoencoders (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
