Research map / Architectures and efficiency / Pruning and efficient tuning
Earning Extra Performance From Restrictive Feedbacks
Jing Li, Yuangang Pan, Yueming Lyu, Yinghua Yao, Yulei Sui, Ivor Wai-Hung Tsang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add layer importance based query reassignment to layerwise tuning (LCPS) (ablation) | LCPS · CIFAR-10-C Gaussian | queries to reach 22.0% error: ~3.0K -> ~0.2K | Improved |
| add layer importance based query reassignment to layerwise tuning (LCPS) (ablation) | LCPS · CIFAR-10-C Impulse | queries to reach 20.0% error: >8.4K -> ~3.5K | Improved |
| use layerwise LCPS instead of whole-model PPS (ablation) | CIFAR-10-C tuning · CIFAR-10-C Gaussian | queries to reach 22.0% error: >10.0K -> ~3.0K | Improved |
| use LCPS with feedback only instead of Tent | corrupted CIFAR-10/100 · CIFAR-10-C average error (Q=1K) | error | Improved |
| use PPS instead of Random Search | Adult income classification · Adult (Q=1K) | accuracy | Improved |
| tune with top-5 error as feedback metric instead of top-1 (ablation) | LCPS · CIFAR-10-C Gaussian and Impulse | top-5 error | Improved |
| round feedback to 0 decimals instead of more decimals (ablation) | PPS · Adult | support accuracy | Hurt |
| increase feedback precision from 2 to 3 decimals (ablation) | PPS · Adult | support accuracy | No clear effect |
| increase support set ratio from 10% to 90% (ablation) | PPS · Adult | holdout accuracy | Improved |
| vary sampling batch size from 2 to 80 (ablation) | PPS · Adult | accuracy: optimum at b=10 support, b=20 holdout | Mixed |
| tune with estimated full-batch gradient (LCPS) instead of standard stochastic-gradient tuning | BERT variants · STS-B | Pearson correlation | Improved |
Closest papers by meaning
- Curvature Tuning: Provable Training-free Model Steering From a Single Parameter (NeurIPS 2025 · similarity 0.84)
- Gatekeeper: Improving Model Cascades Through Confidence Tuning (NeurIPS 2025 · similarity 0.84)
- Sheared Backpropagation for Fine-tuning Foundation Models (CVPR 2024 · similarity 0.83)
- Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How (ICLR 2024 · similarity 0.83)
- LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning (NeurIPS 2024 · similarity 0.83)
- Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
