Research map / Probabilistic and scientific modeling / Differential equations and PDEs

SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models

NeurIPS 2026 · Probabilistic and scientific modeling › Differential equations and PDEs · approach: Physics-informed operator learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SAFE-SVD, a sensitivity-aware SVD compression framework for physics foundation models that combines Fisher-weighted (Sobolev-informed) layer sensitivity, intra-/cross-layer error propagation, and greedy global rank allocation, evaluated on Poseidon, VICON and MPP.

Problem

Compressing physics foundation models (PFMs) while preserving physical fidelity (derivatives, conservation) is under-explored; conventional compression methods degrade or fail on PDE surrogate outputs.

Approach

Post-training SVD compression that weights layer perturbations by a Fisher/Cholesky-whitened loss sensitivity including a Sobolev-space derivative loss, mixes intra-layer and propagated errors via alpha, and allocates ranks globally by greedy singular-value-squared-per-parameter ranking.

Key results

• On Poseidon NS-PwC at compression ratio 0.4, SAFE-SVD keeps Base Loss increase to 0.26% versus 2882%-8741% for baselines.
• At ratio 0.2 on VICON NS-2D, Ours* gives Base Loss increase 239.7% vs 651.3%-15583% for baselines, while Dobi-SVD is best on Euler-2D Sobolev (7.02%).
• On MPP Shallow Water at ratio 0.2, Dobi-SVD raises Base Loss by 27,244.64% while Ours and Ours* limit it to 5,250.31% and 1,330.00%.

Surprising findings

• Using an l1 norm variant of the sensitivity term empirically outperformed the l2 formulation that matches the theory.
• Dobi-SVD and other baselines are sometimes better on specific Sobolev metrics (e.g., SVD-LLM V2 and Dobi-SVD on Euler-2D), though with much higher base loss.

Limitations

• Method requires PDE-family-specific Sobolev loss terms, so it is specific to PDE families.
• Compression-induced errors may limit use in high-precision applications.
• Some baselines outperform on specific Sobolev metrics; Wave-Gaussian at ratio 0.2 shows larger Sobolev error (151.9%).

Experimental setup

Datasets: NS-PwC CE-RM Wave-Gauss NS-BB CE-RPUI Wave-Layer NS-2D Comp-2D Euler-2D IncompNS CompNS DiffRe-2D

Benchmarks: NS-PwC CE-RM Wave-Gauss NS-BB CE-RPUI Wave-Layer NS-2D Comp-2D Euler-2D IncompNS CompNS DiffRe-2D

Metrics: Base Loss (relative % change) Sobolev loss (relative % change) Div-Free error Vorticity error

Baselines: FWSVD ASVD SVD-LLM V2 Dobi-SVD SAES-SVD

Builds on: SVD-LLM FWSVD ASVD Dobi-SVD SAES-SVD

Closest papers by meaning

• ESPACE: Dimensionality Reduction of Activations for Model Compression (NeurIPS 2024 · similarity 0.86)
• In defense of parameter sharing for model-compression (ICLR 2024 · similarity 0.86)
• Forget the Data and Fine-Tuning! Just Fold the Network to Compress (ICLR 2025 · similarity 0.85)
• Unified Scaling Laws for Compressed Representations (NeurIPS 2025 · similarity 0.85)
• Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives (ICLR 2025 · similarity 0.85)
• MeLLoC: Lossless Compression with High-order Mechanism Learning (NeurIPS 2024 · similarity 0.85)