Research map / Vision-language and multimodal / Cross-modal learning
FuseTeacher: Modality-fused Encoders are Strong Vision Supervisors
Chen-Wei Xie, Siyang Sun, Liming Zhao, Pandeng Li, Shuailei Ma, Yun Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add FuseTeacher modality-fused distillation to CLIP | CLIP · YFCC15M-Cap avg of 6 ImageNet variants + retrieval | average (%): 36.4 -> 38.4 | Improved |
| add FuseTeacher on top of MaskCLIP | MaskCLIP · YFCC15M-Cap avg | average (%): 38.3 -> 40.3 | Improved |
| add classification distillation (ablation) | FuseTeacher · ImageNet-1K val zero-shot | zero-shot accuracy (%): 41.1 -> 42.9 | Improved |
| add retrieval distillation (ablation) | FuseTeacher · ImageNet-1K val zero-shot | zero-shot accuracy (%): 41.1 -> 43.0 | Improved |
| combine classification and retrieval distillation (ablation) | FuseTeacher · ImageNet-1K val linear probe | linear-probe accuracy (%): 67.6 -> 68.4 | Improved |
| use fusion encoder embedding instead of image-only embedding for multimodal queries (ablation) | CLIP · ImageNet-MM top-1 | top-1 accuracy (%): 65.2 -> 78.3 | Improved |
| scale backbone from ViT-B/32 to ViT-B/16 with FuseTeacher on Union23M (ablation) | FuseTeacher · Union23M avg | average (%): 53.7 -> 60.6 | Improved |
| apply FuseTeacher alone versus MaskCLIP with ViT-L/14 | FuseTeacher · Union23M ViT-L/14 avg | average (%): 65.9 -> 64.3 | Hurt |
| scale pretraining data from Union23M to Union65M (ablation) | FuseTeacher+ ViT-L/14 · average of ImageNet variants + retrieval | average (%): 67.0 -> 69.9 | Improved |
| use FuseTeacher instead of MaskCLIP for COCO image-to-text retrieval | FuseTeacher · MS-COCO zero-shot retrieval | Image-to-text R@1 (%): 49.4 -> 48.4 | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- MaskCLIP — MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining (CVPR 2023) · better on 6 of 7 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Multi-Modal Representation Learning With Text-Driven Soft Masks (CVPR 2023 · similarity 0.89)
- ITO: Multi-View Alignment and Training-Time Fusion for Image-Text Pretraining (NeurIPS 2026 · similarity 0.89)
- Align before Fuse: Vision and Language Representation Learning with Momentum Distillation (NeurIPS 2021 · similarity 0.88)
- Image Fusion via Vision-Language Model (ICML 2024 · similarity 0.86)
- UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.86)
- MMRL: Multi-Modal Representation Learning for Vision-Language Models (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
