Research map / Architectures and efficiency / Pruning and efficient tuning

Resource-Aware Parameter-Efficient Model Adaptation for Onboard High-Dimensional Data

NeurIPS 2026 · Architectures and efficiency › Pruning and efficient tuning · approach: Low-rank compression and merging · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes NE-LoRA, a LoRA variant with a GELU-activated auxiliary low-rank branch and differentiated per-matrix learning rates for updating onboard hyperspectral classifiers under limited LEO uplink bandwidth.

Problem

Updating hyperspectral satellite models onboard is hard because LEO uplink bandwidth is limited (about 12 Mbps measured) and high-dimensional spectral-spatial inputs make models and updates large.

Approach

Augment LoRA with a GELU-activated auxiliary low-rank branch that is folded into an equivalent low-rank update before upload, plus a differentiated learning-rate scheme (larger for output-side matrices, reduced for the largest-norm layer) to handle asymmetric initialization and gradient dynamics.

Key results

• On CNN3D, NE-LoRA reaches 87.43% OA on HR-L versus 84.93% for LoRA and 81.89% for full fine-tuning.
• NE-LoRA reaches 97.42% OA on Pavia University with CNN3D, above full fine-tuning's 96.90%.
• NE-LoRA's upload size equals LoRA's (e.g., 27.7k vs 161.7k full-model parameters on SA with CNN3D, a 5.8x reduction), though it trains roughly twice as many parameters as LoRA.

Surprising findings

• VeRA, despite updating very few parameters, performs far worse than no fine-tuning on several settings (e.g., 90.55% vs 91.98% OA on SA with CNN3D).
• The NE-LoRA ablation shows the GELU branch alone is not always best; GELU + lambda without gamma wins on WHU-LK for CNN3D and SA for HybridSN.

Limitations

• Evaluation is limited to lightweight 3D CNN hyperspectral classifiers, not larger or general pretrained architectures.
• No error bars or statistical significance reported due to time limits.
• Does not study long-term continual updating under time-varying connectivity or system-level scheduling.

Experimental setup

Datasets: Salinas Valley Pavia University HyRANK-Loukia WHU-LongKou

Benchmarks: Salinas Valley Pavia University HyRANK-Loukia WHU-LongKou

Metrics: Overall Accuracy Average Accuracy Kappa coefficient total parameters trainable parameters upload parameters

Baselines: No Fine-tuning Full Fine-tuning LoRA LoRA+ VeRA

Builds on: LoRA LoRA+ VeRA Kaiming initialization GELU

Closest papers by meaning

• Meta-Learning Hyperparameters for Parameter Efficient Fine-Tuning (CVPR 2025 · similarity 0.83)
• HyperFree: A Channel-adaptive and Tuning-free Foundation Model for Hyperspectral Remote Sensing Imagery (CVPR 2025 · similarity 0.83)
• Parameter Efficient Self-Supervised Geospatial Domain Adaptation (CVPR 2024 · similarity 0.82)
• The Primacy of Magnitude in Low-Rank Adaptation (NeurIPS 2025 · similarity 0.82)
• Learning De-Biased Representations for Remote-Sensing Imagery (NeurIPS 2024 · similarity 0.81)
• LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift (NeurIPS 2026 · similarity 0.81)