Research map / Vision-language and multimodal / Cross-modal learning
Isotropic Embedding Perturbations for Robust Vision Language Encoders
Hyesong Choi, Daeun Kim, Song Park, Taekyung Kim, Byeongho Heo, Sangdoo Yun, Dongbo Min, Dongyoon Han
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| inject perturbation in the embedding space instead of pixel space (ablation) | ViT-B fine-tuning · CUB | accuracy: 79.33 -> 80.89 | Improved |
| inject perturbation in the embedding space instead of pixel space (ablation) | ViT-B fine-tuning · ImageNet-1K | top-1 accuracy: 80.17 -> 82.25 | Improved |
| inject perturbation in the embedding space instead of pixel space (ablation) | ViT-B fine-tuning · token perturbation covariance | anisotropy ratio lambda_max/lambda_min: 8.4 -> 1.15 | Improved |
| add Aether on top of the standard recipe R_b | ViT-B · ImageNet-1K | top-1 accuracy: 81.17 -> 82.25 | Improved |
| add AugMix or RandErase on top of R_b | ViT-B · ImageNet-1K | top-1 accuracy: 81.17 -> 81.16 | No clear effect |
| add Aether on top of R_b | CLIP · ImageNet-1K | top-1 accuracy: 84.10 -> 84.37 | Improved |
| add Aether to fine-tuning | AIMv2 · ImageNet-1K | top-1 accuracy: 86.22 -> 87.01 | Improved |
| add Aether to fine-tuning | SigLIP 2 · ImageNet-1K | top-1 accuracy: 73.64 -> 73.79 | No clear effect |
| add Aether on top of R_b | SwinV2-L · ImageNet-1K | top-1 accuracy: 85.21 -> 85.30 | No clear effect |
| add Aether | ResNet-50 · ImageNet-1K | top-1 accuracy: 79.86 -> 80.04 | No clear effect |
| add Aether on top of R_b in fine-tuning | ViT-L · ImageNet-1K | top-1 accuracy: 84.71 -> 85.35 | Improved |
| add Aether during fine-tuning of diffusion-based SSL model | DiffMAE · ImageNet-1K | top-1 accuracy: 82.18 -> 82.50 | Improved |
Closest papers by meaning
- LORE: Lagrangian-Optimized Robust Embeddings for Visual Encoders (NeurIPS 2025 · similarity 0.87)
- DH-Set: Improving Vision-Language Alignment with Diverse and Hybrid Set-Embeddings Learning (CVPR 2025 · similarity 0.85)
- MMAR: Towards Lossless Multi-Modal Auto-Regressive Probabilistic Modeling (CVPR 2025 · similarity 0.85)
- Test-Time Spectrum-Aware Latent Steering for Zero-Shot Generalization in Vision-Language Models (NeurIPS 2025 · similarity 0.85)
- Exploiting the Asymmetric Uncertainty Structure of Pre-trained VLMs on the Unit Hypersphere (NeurIPS 2025 · similarity 0.85)
- Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients (CVPR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
