Research map / Vision-language and multimodal / Human and embodied modeling
Improving Visual Representation Learning Through Perceptual Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace MAE pixel-only MSE with MS-SSIM + L1 perceptual reconstruction loss (ablation) | MAE · ImageNet-1K linear probe | top-1 accuracy: 67.8 -> 71.2 | Helped |
| Add LS-GAN-P feature-matching perceptual and adversarial loss (ablation) | MAE · ImageNet-1K linear probe | top-1 accuracy: 67.8 -> 72.5 | Helped |
| Use StyleGANv2-ADA training with perceptual loss and multi-scale encoder connections | MAE · ImageNet-1K linear probe | top-1 accuracy: 67.8 -> 78.1 | Helped |
| Use a pre-trained dVAE as the perceptual loss network (ablation) | MAE · ImageNet-1K linear probe | top-1 accuracy: 78.1 -> 79.8 | Helped |
| Add perceptual loss to MSG-GAN adversarial and reconstruction training (ablation) | MSG-MAE · ImageNet-1K fine-tuning, ViT-B | top-1 accuracy: 84.7 -> 85.3 | Helped |
| Add perceptual loss to MSG-GAN adversarial and reconstruction training (ablation) | MSG-MAE · ImageNet-1K fine-tuning, ViT-L | top-1 accuracy: 86.5 -> 87.2 | Helped |
| Add perceptual loss to adversarial and reconstruction training (ablation) | MSG-MAE · ImageNet-1K fine-tuning, ViT-B | top-1 accuracy: 83.3 -> 85.3 | Helped |
| Use multi-scale GAN loss without encoder-decoder skip connections (ablation) | MAE · ImageNet-1K fine-tuning, ViT-B | top-1 accuracy: 85.3 -> 83.2 | Hurt |
| Add multi-scale encoder-decoder skip connections to share gradients with the encoder (ablation) | MAE with MSG-GAN-P · MS COCO | mAP Box: 50.9 -> 52.3 | Helped |
| Replace MSG-GAN-P with StyleGANv2-ADA-P (ablation) | MSG-MAE · ImageNet-1K fine-tuning, ViT-B | top-1 accuracy: 85.3 -> 86.2 | Helped |
| Train MAE longer to match StyleGANv2-ADA-P total training time (ablation) | MAE · ImageNet-1K downstream performance | performance | No clear effect |
| Use LS-GAN without a perceptual loss term (ablation) | MAE · ImageNet-1K downstream performance | performance | Hurt |
Closest papers by meaning
- Closest Neighbors are Harmful for Lightweight Masked Auto-encoders (CVPR 2025 · similarity 0.89)
- Contrastive Tuning: A Little Help to Make Masked Autoencoders Forget (AAAI 2024 · similarity 0.88)
- Exploring Target Representations for Masked Autoencoders (ICLR 2024 · similarity 0.88)
- Perception Encoder: The best visual embeddings are not at the output of the network (NeurIPS 2025 · similarity 0.87)
- Improved Baselines with Representation Autoencoders (NeurIPS 2026 · similarity 0.87)
- The Dynamic Duo of Collaborative Masking and Target for Advanced Masked Autoencoder Learning (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.