Research map / Architectures and efficiency / Pruning and efficient tuning

Minimal Interaction Seperated Tuning: A New Paradigm for Visual Adaptation

CVPR 2025 · Architectures and efficiency › Pruning and efficient tuning · approach: Low-rank compression and merging · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes separated tuning, where a frozen cloud ViT provides a summed intermediate-feature representation to a low-resource device that fine-tunes a small low-rank attention edge network (MIST).

Problem

Fine-tuning large vision pretrained models on low-resource devices is infeasible, and existing side-tuning methods transfer too many intermediate features from the cloud backbone to the device.

Approach

Replace the full set of intermediate features used by side-tuning with their sum (Gather*), which is the external feature term from the skip-connection analysis, cutting transfer to about 1/(N+1) of the original, and pair it with a lightweight low-rank attention edge network.

Key results

• MIST reaches 76.7% average accuracy on VTAB-1K with 0.38M trainable parameters and about 0.58 MB transfer per image, comparable to or above PEFT baselines such as FacT (75.6%) and SPT-LoRA (76.4%).
• On ImageNet domain generalization with 16 shots per class, MIST reaches 76.5 on ImageNet and 37.9 on ImageNet-Sketch, beating NOAH by 5.0 and 13.1 points.
• On a Raspberry Pi 4B with 2GB RAM, fine-tuning peaked at 663 MB memory with batch size 32.

Measured effects reported in the paper

Surprising findings

• Simply adding intermediate features to a linear probe gives only a 1.7% gain over linear probing, while adding the LAE edge network alone gives 9.1%, and combining both is needed for the best result.

Limitations

• Experiments are mostly restricted to visual recognition; segmentation is only briefly covered in the appendix.
• The layer cutoff k must be chosen per task via a validation set from a small candidate set {N, 3N/4, N/2}.
• Gather function is limited to summation; other gather functions are left for future work.

Experimental setup

Datasets: VTAB-1K Aircraft Food101 Pets Flowers102 Cars ImageNet-1K

Benchmarks: VTAB-1K Fine-grained few-shot benchmark (Aircraft, Food101, Pets, Flowers102, Cars) ImageNet-1K domain generalization (ImageNet-V2, Sketch, A, R)

Metrics: top-1 accuracy group-wise average accuracy trainable parameters MACs GPU memory transfer overhead (MB/img)

Baselines: Full fine-tuning BitFit VPT LoRA AdaptFormer FacT SPT-LoRA Linear probing Partial-1 LST LAST NOAH

Builds on: LST LAST side-tuning AdaptFormer VPT

Closest papers by meaning

• 5%>100%: Breaking Performance Shackles of Full Fine-Tuning on Visual Recognition Tasks (CVPR 2025 · similarity 0.88)
• Skip Tuning: Pre-trained Vision-Language Models are Effective and Efficient Adapters Themselves (CVPR 2025 · similarity 0.87)
• ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models (CVPR 2025 · similarity 0.87)
• CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.86)
• pMoE: Prompting Diverse Experts Together Wins More in Visual Adaptation (ICLR 2025 · similarity 0.86)
• VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding (AAAI 2024 · similarity 0.86)