Research map / Probabilistic and scientific modeling / Variational and generative sampling
Spectral Bridge Variational Inference: Dynamic LoRA via Bures-Wasserstein Gradient Flows
Yuhang Xi, Yu-Feng Yu, Chuan-Xian Ren, Zhao-Rong Lai
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace GOAT's 8-expert MoE with a single SBVI adapter | LLaMA-2-7B · GSM8K | accuracy: 60.20 -> 60.85 | Improved |
| use dynamic spectral friction instead of static SVD init (LoRA-One) | LLaMA-2-7B · GSM8K | accuracy: 60.44 -> 60.85 | Improved |
| use SBVI instead of LoRA-One | LLaMA-2-7B · HumanEval | pass@1: 28.66 -> 28.54 | No clear effect |
| use SBVI instead of standard LoRA | LLaMA-2-7B · GSM8K | accuracy: 52.84 -> 60.85 | Improved |
| use SBVI instead of GOAT on commonsense tasks | LLaMA-2-7B · Commonsense170k average | accuracy: 82.73 -> 83.22 | Improved |
| use SBVI instead of LoRA-One | Mistral-7B · GSM8K | accuracy: 70.18 -> 71.24 | Improved |
| use SBVI instead of LoRA-One | LLaMA-3-8B · GSM8K | accuracy: 75.08 -> 76.34 | Improved |
| use SBVI instead of GOAT for memory | LLaMA-2-7B · Training peak memory | peak GPU memory (GB): 34.9 -> 19.8 | Improved |
| use SBVI instead of standard LoRA r=64 for memory | LLaMA-2-7B · Training peak memory | peak GPU memory (GB): 22.8 -> 19.8 | Improved |
| initialize SVD-based PiSSA instead of standard LoRA | LLaMA-2-7B · Commonsense170k average | accuracy: 77.61 -> 73.78 | Hurt |
| vary initial rank r_init among 32, 64, 256 with SBVI (ablation) | SBVI · effective rank | effective rank: reff ≈7.0 ± 0.5 | No clear effect |
| use SBVI instead of MoLoRA | LLaMA-2-7B · HumanEval | pass@1: 24.88 -> 28.54 | Improved |
Closest papers by meaning
- Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation (NeurIPS 2026 · similarity 0.86)
- The Primacy of Magnitude in Low-Rank Adaptation (NeurIPS 2025 · similarity 0.86)
- Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models (ICML 2024 · similarity 0.86)
- VISA: Variational Inference with Sequential Sample-Average Approximations (NeurIPS 2024 · similarity 0.86)
- SING: SDE Inference via Natural Gradients (NeurIPS 2025 · similarity 0.86)
- LoRA Training Provably Converges to a Low-Rank Global Minimum Or It Fails Loudly (But it Probably Won't Fail) (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
