Research map / Vision-language and multimodal / Human and embodied modeling
RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Improves multi-teacher agglomerative vision foundation models (RADIOv2.5) via multi-resolution training, mosaic augmentation, PHI-S teacher balancing, and ToMe token merging for VLMs.
Problem
Agglomerative (multi-teacher distilled) vision foundation models suffer from resolution-dependent mode switching, teacher loss imbalance, teacher artifacts, and too many output tokens when used in VLMs.
Approach
Trains the student against all teachers at multiple resolutions (with feature downsampling and mosaic augmentation for high-res teachers), balances teacher losses via PHI-S standardization, and compresses visual tokens with bipartite token merging instead of pixel unshuffling.
Key results
- Ablation: multi-resolution training gives +2.8 on ADE20k and +3.4 average on VLM benchmarks; SigLIP replacement adds 6.7 points on VLM benchmarks; token merging adds 5.9 average VLM points at 256 tokens vs 576.
- In VILA with MN-Minitron-8B at 196 tokens, RADIOv2.5-H averages 63.19 vs 60.95 for SigLIP-SO400M and 58.55 for OpenAI-CLIP.
- On NYUDv2 and Pascal Context, RADIOv2.5-H reaches 53.97 ADE20k mIoU (linear probe) and RADIOv2.5-g reaches 54.56, outperforming UNIT and UNIC baselines of similar size.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-resolution training (ablation) | RADIOv2.1-L · ADE20k | mIoU: +2.8 on ADE20k | Helped |
| add multi-resolution training (ablation) | RADIOv2.1-L · VLM benchmarks | average score: +3.4 on average on VLMs | Helped |
| replace OpenAI CLIP teacher with SigLIP (ablation) | RADIOv2.5-L multi-resolution · VLM benchmarks | average score: 6.7-point improvement | Helped |
| replace ViT-L backbone with ViT-H (ablation) | RADIOv2.5-L without OpenAI CLIP and with SigLIP · all reported metrics | benchmark scores | Helped |
| replace pixel unshuffling with token merging (ablation) | RADIOv2.5-H · VLM benchmarks | average score: average of 5.9 points | Helped |
| use token merging instead of pixel unshuffling (ablation) | RADIOv2.5-H · TextVQA, ChartQA, DocVQA, InfoVQA, OCRBench | benchmark scores | Helped |
| use mosaic augmentation to generate SAM training inputs (ablation) | RADIOv2.5 student training · student quality | quality | Helped |
Surprising findings
- Token merging yields a larger relative gain on RADIOv2.5 than on SigLIP, attributed to SigLIP's higher feature noise making clustering more error-prone.
Limitations
- Multi-resolution training for 600k iterations is costly, requiring staged training.
- VLM evaluation uses a single LLM (MN-Minitron-8B) and a single data mixture.
Experimental setup
Datasets: DataComp1B ImageNet-1k ADE20k COCO NYUDv2 Pascal Context Navi ShareGPT4V VFLAN LLaVA1.6 mixture
Benchmarks: ImageNet-1k ADE20k COCO Probe3D NYUDv2 Pascal Context TextVQA ChartQA DocVQA InfoVQA OCRBench GQA
Metrics: top-1 accuracy k-NN accuracy mIoU RMSE mErr VLM benchmark accuracy scale equivariance
Baselines: AM-RADIO SAM-CLIP Theia UNIC UNIT OpenAI-CLIP SigLIP-SO400M
Builds on: AM-RADIO PHI-S ToMeSD CPE VILA
Closest papers by meaning
- AM-RADIO: Agglomerative Vision Foundation Model Reduce All Domains Into One (CVPR 2024 · similarity 0.90)
- Masking Teacher and Reinforcing Student for Distilling Vision-Language Models (CVPR 2026 · similarity 0.88)
- Data-efficient Large Vision Models through Sequential Autoregression (ICML 2024 · similarity 0.86)
- DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teachers (CVPR 2025 · similarity 0.86)
- Leveraging Vision-Language Models for Improving Domain Generalization in Image Classification (CVPR 2024 · similarity 0.86)
- LR0.FM: Low-Res Benchmark and Improving robustness for Zero-Shot Classification in Foundation Models (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.