Research map / Architectures and efficiency / Pruning and efficient tuning
The Geometry of Narrow Fine-Tuning Degradation: Trajectory Lock-in and Spectral Bifurcation
Jia Liu, Jiaxin Luo, Xinhao Qiu, Yixue Hao, Min Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SVD spectral regularizer to LoRA fine-tuning | Baseline LoRA · Narrow LoRA fine-tuning | final broad loss: 4.54 -> 3.42 | Improved |
| add SVD spectral regularizer to LoRA fine-tuning | Baseline LoRA · Narrow LoRA fine-tuning | mean Abs-Harm: 0.020 -> 0.012 | Improved |
| add SVD spectral regularizer to LoRA fine-tuning | Baseline LoRA · Narrow LoRA fine-tuning regime comparison | final broad loss: 5.10 -> 3.51 | Improved |
| add SVD spectral regularizer to LoRA fine-tuning | Baseline LoRA · Narrow LoRA fine-tuning regime comparison | final task loss: 0.28 -> 0.31 | No clear effect |
| add gradient noise injection to LoRA fine-tuning | Baseline LoRA · Narrow LoRA fine-tuning regime comparison | final broad loss: 5.10 -> 2.99 | Improved |
| add gradient noise injection to LoRA fine-tuning | Baseline LoRA · Narrow LoRA fine-tuning regime comparison | final task loss: 0.28 -> 0.97 | Hurt |
| add SVD regularizer overhead vs baseline | Baseline LoRA · Training overhead | time per step (ms): 328.7 -> 350.4 | Hurt |
| add null-space projection to LoRA fine-tuning | Baseline LoRA · Training overhead | time per step (ms): 328.7 -> 382.1 | Hurt |
| increase SVD regularization strength s (weak to moderate to very strong) (ablation) | SVD regularizer · Narrow LoRA fine-tuning | broad loss at matched task loss | Mixed |
| delay SVD rescue to later in training (ablation) | SVD regularizer · Narrow LoRA fine-tuning | protection rate | Hurt |
| apply SVD regularization in micro-dataset full fine-tuning (N=64) | SVD regularizer · Full fine-tuning N=64 | broad loss | No clear effect |
Closest papers by meaning
- The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety (NeurIPS 2026 · similarity 0.90)
- The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions (ICML 2025 · similarity 0.87)
- Gradient Descent as Loss Landscape Navigation: a Normative Framework for Deriving Learning Rules (NeurIPS 2025 · similarity 0.87)
- The Sharp Directions Are Against You: Curvature Analysis of Activation Steering (NeurIPS 2026 · similarity 0.86)
- LoRA Training Provably Converges to a Low-Rank Global Minimum Or It Fails Loudly (But it Probably Won't Fail) (ICML 2025 · similarity 0.86)
- Curvature Tuning: Provable Training-free Model Steering From a Single Parameter (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
