Research map / Architectures and efficiency / Pruning and efficient tuning
Enhancing Low-Rank Adaptation with Recoverability-Based Reinforcement Pruning for Object Counting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes E3RP, which uses a DQN agent with a recoverability reward (cosine feature similarity plus a learned action classifier) to prune SAM's parameters globally and fine-tunes a scale-aware counting head with Patch Query Flip Attention for object counting.
Problem
Large vision models like SAM carry redundant parameters for object counting, and transferring them with full fine-tuning or LoRA is costly and may hurt performance across scenarios with distribution gaps.
Approach
Use a deep Q-learning agent to pick the global L1 unstructured pruning ratio of a frozen large model, with a reward that combines cosine similarity of pre/post-pruning features and a learned keep/prune classifier, followed by fine-tuning a decoder (SACH with PQFA) for counting.
Key results
- E3RP achieves MAE 4.18 / MSE 5.51 on CARPK and MAE 1.40 / MSE 1.85 on PUCPR, state of the art on the two drone datasets in Table 1.
- E3RP prunes up to 59% of SAM-B parameters on SHHB (MAE 6.46, MSE 10.39) and 53% on PUCPR while reducing error versus unpruned SAM-B.
- Pruned SAM-B with DQN-P and LoRA reaches MAE 1.40 on PUCPR versus 1.90 for SAM-B+LoRA with more total parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use E3RP instead of HKINet | object counting · CARPK | MAE: 4.89 -> 4.18 | Helped |
| Use E3RP instead of BDRNet | object counting · PUCPR | MAE: 1.91 -> 1.40 | Helped |
| Use E3RP instead of PSGCNet | object counting · SHHA | MAE: 56.10 -> 58.37 | Hurt |
| Use E3RP instead of PSGCNet | object counting · SHHB | MAE: 6.60 -> 6.46 | Helped |
| Add DQN-P (ablation) | SAM-B counting model · SHHB | MAE: 6.96 -> 6.90 | Helped |
| Add PQFA (ablation) | SAM-B counting model · SHHB | MAE: 6.96 -> 6.61 | Helped |
| Add DQN-P and PQFA (ablation) | SAM-B counting model · SHHB | MAE: 6.96 -> 6.46 | Helped |
| Add LoRA without DQN-P (ablation) | SAM-B · PUCPR | MAE: 2.02 -> 1.90 | Helped |
| Add DQN-P without LoRA (ablation) | SAM-B · PUCPR | MAE: 2.02 -> 1.67 | Helped |
| Add DQN-P and LoRA (ablation) | SAM-B · PUCPR | MAE: 2.02 -> 1.40 | Helped |
| Lower similarity threshold from τ=0.99 to τ=0.98 at r=1 (ablation) | DQN-P · PUCPR | MAE: 1.47 -> 2.33 | Hurt |
| Lower similarity threshold from τ=0.99 to τ=0.97 at r=1 (ablation) | DQN-P · PUCPR | Pruning Ratio: 53% -> 63% | Helped |
Surprising findings
- Removing parameters from SAM-B via pruning improved counting error over the unpruned model in the reported ablations.
Limitations
- Does not achieve best results on every dataset; some baselines are competitive.
- Excessive reduction of the similarity threshold tau degrades performance.
- Evaluated only on SAM-B and counting datasets; extension to multimodal large models is future work.
Experimental setup
Datasets: CARPK PUCPR SHHA SHHB
Benchmarks: CARPK PUCPR SHHA SHHB
Metrics: MAE MSE
Baselines: MCNN CSRNet BL ASPDNet PSGCNet LMSFFNet BDRNet HKINet LoRA
Builds on: LoRA DQN SAM
Closest papers by meaning
- T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting (CVPR 2025 · similarity 0.83)
- Learning to Count without Annotations (CVPR 2024 · similarity 0.83)
- Unified Knowledge Maintenance Pruning and Progressive Recovery with Weight Recalling for Large Vision-Language Models (AAAI 2025 · similarity 0.83)
- VLCounter: Text-Aware Visual Representation for Zero-Shot Object Counting (AAAI 2024 · similarity 0.83)
- Unveiling the Visual Counting Bottleneck in Vision-Language Models (ICML 2026 · similarity 0.83)
- Understanding Counting Mechanisms in Large Language and Vision-Language Models (CVPR 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.