Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
GPan-LoRA: Gaussian Process Amortized Networks for Bayesian Low-Rank Adaptation in Large Language Models
Weifeng Zhang, Wenyuan Zhao, Amir Hossein Rahmati, Yucheng Wang, Zhiyuan Wang, Chao Tian, Xiaoning Qian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add amortized network (ablation) | GPan-LoRA · OBQA->Chem (large shift) | ECE (%): 22.60 -> 11.31 | Improved |
| add amortized network (ablation) | GPan-LoRA · OBQA->Phy (large shift) | ECE (%): 26.15 -> 13.29 | Improved |
| add amortized network (ablation) | GPan-LoRA · OBQA->Chem (large shift) | NLL: 1.37 -> 1.19 | Improved |
| add amortized network (ablation) | GPan-LoRA · OBQA (in-distribution) | accuracy (%): 87.27 -> 86.66 | No clear effect |
| add weight sharing (ablation) | GPan-LoRA · OBQA->Phy (large shift) | ECE (%): 22.52 -> 13.29 | Improved |
| add weight sharing (ablation) | GPan-LoRA · OBQA (in-distribution) | accuracy (%): 84.73 -> 86.66 | Improved |
| add weight sharing (ablation) | GPan-LoRA · OBQA->Chem (large shift) | NLL: 1.34 -> 1.19 | Improved |
| add amortized network (ablation) | GPan-LoRA · OBQA->ARC-C (small shift) | ECE (%): 6.35 -> 6.46 | No clear effect |
| Bayesianize only the last LoRA layer instead of all layers | GPan-LoRA · OBQA->Chem (large shift) | ECE (%): 11.31 -> 14.37 | Hurt |
| replace BLoB with GPan-LoRA | BLoB · OBQA->Chem (large shift) | ECE (%): 24.45 -> 11.31 | Improved |
| replace LAP with GPan-LoRA | LAP · OBQA->Chem (large shift) | ECE (%): 14.59 -> 11.31 | Improved |
| replace LAP with GPan-LoRA | LAP · OBQA->ARC-E (small shift) | accuracy (%): 88.01 -> 82.74 | Hurt |
Closest papers by meaning
- C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Models (NeurIPS 2025 · similarity 0.90)
- Training-Free Bayesianization for Low-Rank Adapters of Large Language Models (NeurIPS 2025 · similarity 0.89)
- BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models (NeurIPS 2024 · similarity 0.88)
- Dynamic Low-Rank Sparse Adaptation for Large Language Models (ICLR 2025 · similarity 0.88)
- Bayesian Low-rank Adaptation for Large Language Models (ICLR 2024 · similarity 0.88)
- On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
