Research map / Detection and segmentation / Open-set and few-shot detection

AM-RADIO: Agglomerative Vision Foundation Model Reduce All Domains Into One

CVPR 2024 · Detection and segmentation › Open-set and few-shot detection · approach: Open-vocabulary 3D and world perception · 46 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AM-RADIO, a multi-teacher feature distillation framework that trains one vision foundation model (RADIO) from CLIP, DINOv2 and SAM teachers, plus an efficient hybrid backbone E-RADIO.

Problem

Vision foundation models such as CLIP, DINOv2 and SAM each have distinct strengths and gaps, and there is no single backbone combining them; efficient architectures are also hard to compare fairly for foundation-model distillation.

Approach

Distill several heterogeneous VFMs into one student by matching each teacher's summary and spatial features through teacher-specific MLP adaptor heads, with no ground-truth labels, and use CPE and multi-resolution training to let the student handle different teacher input resolutions; also introduces the E-RADIO hybrid CNN-transformer student.

Key results

• RADIO-ViT-H/16 reaches 86.06 ImageNet-1K k-NN and 51.34 ADE20K linear-probe mIoU, outperforming the teachers on 6 of 9 tasks in Table 1.
• E-RADIO-L achieves 83.89 k-NN and 48.22 ADE20K with throughput of 468 vs 63 for InternViT-6B, and is at least 6x faster than teachers at matched resolution.
• Replacing SAM's encoder with RADIO gives COCO mIoU 76.23 (ViTDet-H/16-W16) vs 77.18 for SAM-Huge, at higher throughput (27.91 vs 11.83).

Measured effects reported in the paper

Surprising findings

• Adding a spatial feature loss from teachers improved holistic summary metrics as well as dense tasks.
• Automatic loss balancing (uncertainty, AdaLoss, AMTML-KD) gave little benefit over naive equal weighting; AMTML-KD collapsed toward CLIP.

Limitations

• Latent low- and high-resolution modes in RADIO features cause discontinuities at certain resolutions (e.g. around 720px).
• Adaptor heads are simple MLPs and loss balancing was mostly naive; larger design space left unexplored.
• SAM is not well suited to downstream tasks out of the box, and SAM-distilled features are only partly integrated.

Experimental setup

Datasets: DataComp-1B ImageNet-1K ImageNet-21K LAION-400M ADE20K PascalVOC2012 COCO 2017 GQA TextVQA VQAv2 POPE ScienceQA

Benchmarks: ImageNet-1K k-NN ImageNet-1K zero-shot ADE20K linear probe PascalVOC2012 linear probe COCO 2017 SAM drop-in GQA TextVQA POPE VQAv2

Metrics: top-1 accuracy mIoU throughput VQA accuracy

Baselines: OpenCLIP ViT-H/14 MetaCLIP ViT-H/14 SigLIP-M/14 Intern-ViT-6B DFN CLIP ViT-H/14 OpenAI CLIP ViT-L/14 DINOv2-g/14-reg SAM ViT-H EfficientNetV2-S ConvNext-B SwinV2-S MaxViT-B

Builds on: knowledge distillation CPE LLaVA-1.5 ViTDet EdgeViT

Closest papers by meaning

• RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models (CVPR 2025 · similarity 0.90)
• Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models (NeurIPS 2025 · similarity 0.85)
• General Object Foundation Model for Images and Videos at Scale (CVPR 2024 · similarity 0.84)
• Mamba as a Bridge: Where Vision Foundation Models Meet Vision Language Models for Domain-Generalized Semantic Segmentation (CVPR 2025 · similarity 0.84)
• Unleashing the Power of Visual Foundation Models for Generalizable Semantic Segmentation (AAAI 2025 · similarity 0.83)
• How Well Does GPT-4o Understand Vision? Evaluating Multimodal Foundation Models on Standard Computer Vision Tasks (ICLR 2026 · similarity 0.83)