AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

Mixed Autoencoder for Self-Supervised Visual Representation Learning

Kai Chen, Zhili Liu, Lanqing Hong, Hang Xu, Zhenguo Li, Dit-Yan Yeung

CVPR 2023 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Masked autoencoder pretraining · 35 citations (OpenAlex)

This paper tried a common change: Add a contrastive loss

Add homologous contrastive loss alongside homologous attention · ADE20K, 300-epoch ablation · mIoU: 44.5 -> 46.4 Improved

What 1,317 accepted papers report for this change

See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add homologous recognition to MAE-style mixingMAE with ViT-Base · ImageNet-1K, 300 epochsTop-1 accuracy: 82.7 -> 83.1Improved
Add homologous recognition to MAE-style mixingMAE with ViT-Base · ADE20K, 300 epochsmIoU: 46.1 -> 47.0Improved
Add homologous recognition to MAE-style mixingMAE with ViT-Base · COCO, 300 epochsbox AP: 47.2 -> 47.8Improved
Add homologous recognition to MAE-style mixingMAE with ViT-Base · ImageNet-1K, 1600 epochsTop-1 accuracy: 83.6 -> 83.9Improved
Set the mixing ratio to r=0.25 rather than r=0.5 (ablation)MixedAE · ImageNet-1K, 300-epoch ablationTop-1 accuracy: 82.3 -> 82.4Improved
Set the mixing ratio to r=0.25 rather than r=0.5 (ablation)MixedAE · ADE20K, 300-epoch ablationmIoU: 44.2 -> 45.0Improved
Apply homologous contrastive loss after LayerNorm rather than before it (ablation)MixedAE · ADE20K, 300-epoch ablationmIoU: 44.5 -> 45.3Improved
Use each homologous positive separately instead of averaging positives (ablation)MixedAE · ADE20K, 300-epoch ablationmIoU: 45.1 -> 45.3Improved
Set homologous-attention threshold K to 0.25L instead of disabling TopK sampling (ablation)MixedAE · ADE20K, 300-epoch ablationmIoU: 45.3 -> 45.6Improved
Keep global self-attention in the first layer only (ablation)MixedAE · ADE20K, 300-epoch ablationmIoU: 45.6 -> 46.4Improved
Add homologous contrastive loss alongside homologous attention (ablation)MixedAE · ADE20K, 300-epoch ablationmIoU: 44.5 -> 46.4Improved
Add homologous attention alongside homologous contrastive loss (ablation)MixedAE · ADE20K, 300-epoch ablationmIoU: 45.3 -> 46.4Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy