Research map / Architectures and efficiency / Pruning and efficient tuning
Dropout Mixture Low-Rank Adaptation for Visual Parameters-Efficient Fine-Tuning
Zhengyi Fang, Yue Wang, Ran Yi, Lizhuang Ma
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DMLoRA-LS instead of best prior PEFT method | ViT-B/16 PEFT · VTAB-1K (19 tasks avg) | accuracy: 76.1 -> 77.0 | Improved |
| use DMLoRA-LS instead of SSF on FGVC | ViT-B/16 PEFT · FGVC (5 datasets avg) | accuracy: 90.7 -> 90.7 | No clear effect |
| gradually halve branches during training instead of keeping n branches (ablation) | DMLoRA · VTAB-1K (avg) | accuracy: 74.5 -> 76.6 | Improved |
| prune branches by random dropout instead of parameter averaging (ablation) | DMLoRA · VTAB-1K (avg) | accuracy: 74.8 -> 76.6 | Improved |
| use multiple up and down branches instead of sharing the up matrix (ablation) | DMLoRA · VTAB-1K (avg) | accuracy: 75.2 -> 76.6 | Improved |
| use multiple up and down branches instead of sharing the down matrix (ablation) | DMLoRA · VTAB-1K (avg) | accuracy: 75.5 -> 76.6 | Improved |
| learn the scale in a second stage with frozen LoRA matrices instead of jointly (ablation) | DMLoRA · VTAB-1K (avg) | accuracy: 75.4 -> 77.0 | Improved |
| use DMLoRA-LS vs prior methods on structured VTAB tasks sNORB-Azim and Caltech101 | ViT-B/16 PEFT · VTAB-1K per-task | accuracy: 36.1 vs 38.6 (RepAdapter, sNORB-Azim); 90.7 vs 92.7 (NOAH, Caltech101) | Mixed |
| add reparameterized LoRA branches at inference instead of Adapter/VPT | DMLoRA-LS · ViT-B/16 inference latency | GPU latency (b=128): 1.37x -> 1x (vs VPT-Deep) | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- E2VPT — E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning (ICCV 2023) · better on 1 of 1 shared result
- AdaptFormer — AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition (NeurIPS 2022) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- RandLoRA: Full rank parameter-efficient fine-tuning of large models (ICLR 2025 · similarity 0.88)
- PELA: Learning Parameter-Efficient Models with Low-Rank Approximation (CVPR 2024 · similarity 0.88)
- Low-Rank Rescaled Vision Transformer Fine-Tuning: A Residual Design Approach (CVPR 2024 · similarity 0.88)
- Tuning Stable Rank Shrinkage: Aiming at the Overlooked Structural Risk in Fine-tuning (CVPR 2024 · similarity 0.87)
- Sensitivity-Aware Efficient Fine-Tuning via Compact Dynamic-Rank Adaptation (CVPR 2025 · similarity 0.87)
- Decoupling Angles and Strength in Low-rank Adaptation (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
