Research map / Architectures and efficiency / Pruning and efficient tuning

F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning

NeurIPS 2025 · Architectures and efficiency › Pruning and efficient tuning · approach: Adapter-based parameter-efficient tuning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Systematic study of parameter-efficient fine-tuning for pretrained Fourier-based Large Operator Models, showing bottleneck adapters beat LoRA and proposing F-Adapter, which allocates adapter bottleneck widths per frequency band, reaching SOTA on 3D Navier-Stokes with under 2% extra parameters.

Problem

Parameter-efficient fine-tuning of pretrained large operator models (LOMs) for scientific machine learning, where existing PEFT methods from NLP/CV (notably LoRA) underperform on PDE surrogate tasks.

Approach

Replace linear low-rank updates with residual two-layer MLP adapters inserted into Fourier layers, and allocate bottleneck width per radial frequency band (larger for low frequencies, smaller for high) based on the observation that PDE solution energy concentrates in low frequencies.

Key results

• On 3D Navier-Stokes (M=1.0), F-Adapter reaches L2RE 0.5329 with 1.91% trainable parameters versus 0.6395 for LoRA (r=32, 1.37%) and 0.5496 for the vanilla Adapter.
• On Turbulence, F-Adapter achieves L2RE 0.4523, better than the vanilla Adapter (0.4696) and far better than LoRA (0.6842), though full fine-tuning reaches 0.2382.
• On Poseidon for SWE-2D, the proposed F-LoRA achieves L2RE 0.2746 versus 0.4010 for LoRA, with 4.78% parameters.

Measured effects reported in the paper

Surprising findings

• Standard LoRA does not benefit from increasing rank on DPOT and performs markedly worse than bottleneck adapters.
• Adapters perform strongly on FNO-based backbones but their effectiveness declines sharply on the transformer-based Poseidon backbone, where LoRA is stronger.

Limitations

• Theory relies on a low-frequency energy dominance assumption that may not hold for highly nonlinear multi-physics flows.
• Adapter advantage is reported to decline on transformer backbones (Poseidon), so the method is primarily validated on FNO-based models.
• Experiments are limited to a few PDE datasets; error bars and repeated-run statistics are not reported.

Experimental setup

Datasets: PDEBench 3D Navier-Stokes (Rand, Turbulence, M=1.0 and M=0.1) SWE-2D MHD-3D (24 trajectories)

Benchmarks: PDEBench 3D Navier-Stokes SWE-2D MHD-3D

Metrics: L2 relative error % trainable parameters GPU memory (GB) FLOPs latency (ms)

Baselines: LoRA AdaLoRA HydraLoRA Prompt Tuning Vanilla Adapter FiLM Adapter RandLoRA SVFT Full Fine-Tuning Chebyshev Adapter Fourier Adapter WaveAct Adapter

Builds on: LoRA bottleneck adapter DPOT AdaLoRA HydraLoRA

Closest papers by meaning

• Parameter-Efficient Fine-Tuning with Discrete Fourier Transform (ICML 2024 · similarity 0.89)
• Spectral Adapter: Fine-Tuning in Spectral Space (NeurIPS 2024 · similarity 0.89)
• Linearization Explains Fine-Tuning in Large Language Models (NeurIPS 2025 · similarity 0.88)
• FourierMoE: Fourier Mixture-of-Experts Adaptation of Large Language Models (NeurIPS 2026 · similarity 0.88)
• A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models (ICML 2025 · similarity 0.87)
• The Primacy of Magnitude in Low-Rank Adaptation (NeurIPS 2025 · similarity 0.87)