Research map / Architectures and efficiency / Pruning and efficient tuning
LION: Implicit Vision Prompt Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LION, a lightweight vision prompt tuning method that inserts two equilibrium (DEQ) implicit layers at the input and output of a frozen backbone and prunes/regularizes their parameters via a lottery-ticket-inspired criterion.
Problem
Vision prompt tuning still needs too many trainable parameters from extensive per-layer prompt blocks and complex, tuning-sensitive prompt designs when adapting large vision Transformers and CNNs.
Approach
Replace stacked per-layer prompt MLPs with just two single-layer Deep Equilibrium (implicit) prompt blocks at the input and output of a frozen backbone, blended with adaptive coefficients, and train only the crucial parameters (selected by |gradient × value|) while shrinking non-crucial ones toward zero.</ntml:parameter>
Key results
- On ViT-B, LION reports 0.9077 on CIFAR10 vs. 0.9049 for VPT, with 0.124M trainable parameters vs. 0.523M for VPT.
- On ResNet-50 CIFAR100, LION (0.5484) is below VPT (0.6289) despite the lowest parameter count (0.097M).
- Ablation shows removing the robust training mechanism lowers ImageNet100 accuracy from 0.7372 to 0.7197 on ResNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LION instead of VPT | ViT-B · CIFAR10 | accuracy: 0.9049 -> 0.9077 | Helped |
| Use LION instead of VPT | ViT-B · CIFAR100 | accuracy: 0.6689 -> 0.6541 | Hurt |
| Use LION instead of VPT | ViT-B · ImageNet100 | accuracy: 0.7596 -> 0.7612 | Helped |
| Use LION instead of VPT | ViT-B · Flower | accuracy: 0.9013 -> 0.9054 | Helped |
| Use LION instead of VPT | ViT-B · Dogs | accuracy: 0.8367 -> 0.8361 | Hurt |
| Use LION instead of VPT | ViT-B · Cars | accuracy: 0.7682 -> 0.7991 | Helped |
| Use LION instead of VPT | ViT-B · CIFAR10-LongTail IR100 | accuracy: 0.7631 -> 0.7829 | Helped |
| Use LION instead of VPT | ViT-B · CIFAR100-LongTail IR100 | accuracy: 0.4785 -> 0.5024 | Helped |
| Add the input-side equilibrium layer (ablation) | LION · CIFAR10 | accuracy: 0.8501 -> 0.8628 | Helped |
| Add the output-side equilibrium layer (ablation) | LION · CIFAR10 | accuracy: 0.8239 -> 0.8628 | Helped |
| Add robust training (ablation) | LION · CIFAR10 | accuracy: 0.8516 -> 0.8628 | Helped |
| Add the input-side equilibrium layer (ablation) | LION · CIFAR100 | accuracy: 0.6413 -> 0.6541 | Helped |
Surprising findings
- On CIFAR100 with both ResNet-50 and ResNet-101, LION underperforms VPT, contradicting the claim of uniform superiority.
- Prompts placed only at the output (LION-P2) perform much worse than the full design, indicating high-level activation matters most.
Limitations
- Underperforms VPT on CIFAR100 for CNN backbones and on some few-shot/Dogs tasks.
- Performance gains are modest (often under 1%) and the reported improvements are not accompanied by variance estimates.
- Stacking more DEQ layers gives only slight gains at several times the compute cost.
Experimental setup
Datasets: ImageNet100 CIFAR10 CIFAR100 Flower Stanford Dogs Stanford Cars Clothing Oxford-IIIT Pets Food-101
Benchmarks: CIFAR10 CIFAR100 ImageNet100 Flower Stanford Dogs Stanford Cars Clothing CIFAR10-LongTail CIFAR100-LongTail Pets Food-101
Metrics: accuracy
Baselines: Retraining Head-tuning Fine-tuning Adapter Bias tuning VPT
Builds on: Deep Equilibrium Models Visual Prompt Tuning (VPT) Lottery Ticket Hypothesis
Closest papers by meaning
- DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformers (CVPR 2025 · similarity 0.88)
- Learning to Learn Better Visual Prompts (AAAI 2024 · similarity 0.88)
- LoR-VP: Low-Rank Visual Prompting for Efficient Vision Model Adaptation (ICLR 2025 · similarity 0.88)
- ZIP: An Efficient Zeroth-order Prompt Tuning for Black-box Vision-Language Models (ICLR 2025 · similarity 0.88)
- Black-Box Test-Time Prompt Tuning for Vision-Language Models (AAAI 2025 · similarity 0.87)
- Visual Fourier Prompt Tuning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.