AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

Efficient Low-rank Backpropagation for Vision Transformer Adaptation

Yuedong Yang, Hung-Yueh Chiang, Guihong Li, Diana Marculescu, Radu Marculescu

NeurIPS 2023 · Architectures and efficiency › Pruning and efficient tuning · approach: Efficient ViT adaptation · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use LBP-WHT LP_L1-2 instead of LoRALBP-WHT · EfficientFormer-L1 last stage, CIFAR100accuracy: 75.61 vs 65.25Improved
Use LBP-WHT LP_L1-4 instead of LoRA-allLBP-WHT · EfficientFormer-L1 last stage, CIFAR100accuracy: 78.27 vs 76.92Improved
Use LP_L1-8 rank-36 projection instead of full-rank BPLBP-WHT · EfficientFormer-L1 last stage, CIFAR100accuracy: 79.28 -> 79.34Improved
Reduce rank from LP_L1-8 to LP_L1-4 (ablation)LBP-WHT · EfficientFormer-L1 last stage, CIFAR100accuracy: 79.34 -> 78.27Mixed
Reduce rank from LP_L1-4 to LP_L1-2 (ablation)LBP-WHT · EfficientFormer-L1 last stage, mean accuracymean accuracy: 87.76 -> 85.50Mixed
Use LBP-WHT LP_L1-2 instead of LoRA-allLBP-WHT · SwinV2-Small last stage, mean accuracymean accuracy: 84.86 -> 90.32Improved
Use LBP-WHT LP_L1-4 instead of LoRA-allLBP-WHT · Segformer partial, CityscapesmIoU: 58.07 -> 60.97Improved
Use LBP-WHT LP_L1-4 instead of LoRA-all in full trainingLBP-WHT · Segformer full training, CityscapesmIoU: 65.74 -> 64.72Hurt
Use a small rank (LP_L1-2) in full trainingLBP-WHT · Segformer full training, CityscapesmIoU: 67.37 -> 61.57Hurt
Use LP_L-inf base selection instead of LP_L1 (ablation)LBP-WHT · EfficientFormer-L1 CIFAR100accuracy: 78.27 -> 77.64No clear effect
Use LHE profiling instead of LP_L1 base selection (ablation)LBP-WHT · EfficientFormer-L1 CIFAR100accuracy: 78.27 -> 78.06No clear effect
Use LP_L1 low-frequency WHT bases (ablation)LBP-WHT · EfficientFormer-L1 last attention blockgradient energy preservedImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy