Research map / Continual learning / Continual fine-tuning of LLMs
HiDe-PET: Continual Learning via Hierarchical Decomposition of Parameter-Efficient Tuning
Liyuan Wang, Jingyi Xie, Xingxing Zhang, Hang Su, Jun Zhu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add within-task prediction (WTP) to the naive task-specific baseline (ablation) | HiDe-LoRA · Split Cars-196 (Sup-21/1K) | FAA: 45.39 -> 48.01 | Improved |
| add task-identity inference (TII) to WTP (ablation) | HiDe-LoRA · Split Cars-196 (Sup-21/1K) | FAA: 48.01 -> 49.62 | Improved |
| add task-adaptive prediction (TAP) to WTP (ablation) | HiDe-LoRA · Split Cars-196 (Sup-21/1K) | FAA: 48.01 -> 63.28 | Improved |
| add TII on top of WTP+TAP (ablation) | HiDe-LoRA · Split Cars-196 (Sup-21/1K) | FAA: 63.28 -> 69.65 | Improved |
| add TAP to WTP (ablation) | HiDe-PreT · Split Cars-196 (Sup-21/1K) | FAA: 48.23 -> 61.38 | Improved |
| add TAP to WTP (ablation) | HiDe-Adapter · Split ImageNet-R (Sup-21/1K) | FAA: 76.10 -> 76.98 | Improved |
| use FSA+SL instead of fix-and-tuning for task-shared parameters (ablation) | HiDe-LoRA · Split Cars-196 (Sup-21/1K) | FAA-U: 57.68 -> 62.89 | Improved |
| use FSA+SL instead of EMA for task-shared parameters (ablation) | HiDe-Adapter · Split Cars-196 (Sup-21/1K) | FAA-U: 64.46 -> 64.93 | Improved |
| use variance instead of covariance for representation recovery (ablation) | HiDe-PET · Split ImageNet-R (Sup-21/1K) | FAA | No clear effect |
| replace LAE-LoRA with HiDe-LoRA | LAE-LoRA · Split Cars-196 (Sup-21/1K) | FAA: 50.90 -> 69.65 | Improved |
| replace LAE-LoRA with HiDe-LoRA | LAE-LoRA · Split CUB-200 (iBOT-21K) | FAA: 56.16 -> 76.10 | Improved |
| replace LAE-LoRA with HiDe-LoRA | LAE-LoRA · Split ImageNet-R (Sup-Weak) | FAA: 57.40 -> 59.40 | Improved |
Closest papers by meaning
- SAFE: Slow and Fast Parameter-Efficient Tuning for Continual Learning with Pre-Trained Models (NeurIPS 2024 · similarity 0.91)
- STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.89)
- InTAct: Interval-based Task Activation Consolidation for Continual Learning (NeurIPS 2026 · similarity 0.89)
- Model Sensitivity Aware Continual Learning (NeurIPS 2024 · similarity 0.88)
- Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective (ICML 2025 · similarity 0.88)
- Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
