AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Revisiting Model Stitching In the Foundation Model Era

Zheda Mai, Ke Zhang, Fu-En Wang, Zixiao Ken Wang, Albert Y. C. Chen, Lu Xia, Min Sun, Wei-Lun Chao, Cheng-Hao Kuo

CVPR 2026 · Architectures and efficiency › RNNs and normalization · approach: Layer-wise analysis of language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
pre-train stitch layer with final feature matching before task loss training (ablation)DINOv2->SigLIP2 MLP stitch · fMoW, stitch layer 2accuracy (%): 25.1 -> 51.7Improved
pre-train stitch layer with final feature matching before task loss training (ablation)SigLIP2->DINOv2 MLP stitch · fMoW, stitch layer 2accuracy (%): 38.7 -> 53.8Improved
match final features (FFM) instead of layer features (LFM) at stitch position for initialization (ablation)DINOv2->SigLIP2 · fMoW, stitch layer 2accuracy (%): 33.1 -> 51.7Improved
FFM vs LFM initialization at deep stitch positions (ablation)DINOv2->SigLIP2 and SigLIP2->DINOv2 · fMoW, layers 14-22accuracy (%): 70.3 (LFM) vs 71.8 (FFM) at layer 22No clear effect
use stitch layer trained on layer feature matching only (low layer distance) (ablation)SigLIP->DINOv2 · final feature distanceL2 feature distanceHurt
stitch heterogeneous VFMs instead of self-stitching the source or target (ablation)DINOv2/SigLIP2 stitched models · fMoW stitch layer 22accuracy (%): 68.9 -> 71.8Improved
stitch heterogeneous VFMs instead of self-stitch (ablation)DINOv2/SigLIP2 on ADE20K · ADE20K, layers 14 and 22mIoU (%): +0.5 to +0.7 mIoUImproved
use CLIP as the source model in stitchingstitched VFMs with CLIP source · fMoW, DINOv2/DINOv3/SigLIP2/CLIPaccuracy (%)Hurt
use MLP stitch layer instead of linear layer (ablation)DINOv2->SigLIP2 stitching · fMoW, layer 18accuracy (%): 69.1 -> 72.0Improved
use LoRA stitch layer instead of MLP (ablation)DINOv2->SigLIP2 stitching · fMoW, layer 18accuracy (%): 72.0 -> 67.7Hurt
train stitch layer on task-agnostic LLaVA-1.5 images instead of task-specific data (ablation)FFM stitching, layer 22 · fMoW, iNaturalist linear probingaccuracy (%): fMoW 54.7 -> 53.8; iNat 75.5 -> 77.7 (S to D)No clear effect
share early layers across VFMs (VFM Stitch Tree at layer 14)MoF-LLaVA (CLIP + DINOv2) with Qwen 3B · VQAv2, MMEshare of two-VFM gain recovered: 84% of gain at 39% extra resourcesImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy