Research map / Probabilistic and scientific modeling / Molecules and proteins
Approximately Equivariant Neural Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes approximately equivariant neural processes by adding a small number of learnable fixed inputs to equivariant decoders, with a theoretical result that any Hölder non-equivariant operator can be approximated this way.
Problem
Real-world stochastic processes are only approximately equivariant, yet existing equivariant neural processes enforce strict equivariance, and existing approximate-equivariance methods are tied to specific architectures (CNN/MLP) or loss modifications.
Approach
Approximate any non-equivariant operator between function spaces by an equivariant operator with B additional fixed learnable inputs; implement by adding these inputs to the embedding of an existing equivariant NP decoder, randomly zeroing them during training and zeroing them outside the training domain so predictions revert to equivariant ones OOD.
Key results
- On the 1-D GP task, TNP(T̃) matched non-equivariant TNP ID (-0.406) and matched equivariant TNP OOD (-0.424 vs -0.430 for T).
- On 2-D smoke, TNP(T̃) reached 4.715 log-likelihood versus 4.299 for non-equivariant TNP and 4.181 for TNP(T).
- On 2-D ERA5 Europe, EquivCNP(Ẽ) reached 1.36 vs 1.27 for EquivCNP(E); approximately equivariant models outperformed counterparts in most settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add fixed inputs to the TNP decoder | TNP (T) · Synthetic 1-D GP ID | test log-likelihood: -0.500 \pm 0.004 -> -0.406 \pm 0.004 | Helped |
| Add fixed inputs to the TNP decoder | TNP (T) · Synthetic 1-D GP OOD | test log-likelihood: -0.430 \pm 0.007 -> -0.424 \pm 0.007 | No clear effect |
| Add fixed inputs to the TNP decoder | TNP (T) · 2-D Smoke | test log-likelihood: 4.181 \pm 0.011 -> 4.715 \pm 0.010 | Helped |
| Add fixed inputs to the ConvCNP decoder | ConvCNP (T) · Synthetic 1-D GP ID | test log-likelihood: -0.499 \pm 0.004 -> -0.430 \pm 0.004 | Helped |
| Add fixed inputs to the ConvCNP decoder | ConvCNP (T) · 2-D Smoke | test log-likelihood: 3.637 \pm 0.041 -> 3.827 \pm 0.011 | Helped |
| Use relaxed convolutions | ConvCNP (T) · 2-D Smoke | test log-likelihood: 3.637 \pm 0.041 -> 4.006 \pm 0.010 | Helped |
| Add fixed inputs to the EquivCNP decoder | EquivCNP (E) · 2-D Smoke | test log-likelihood: 4.194 \pm 0.015 -> 4.233 \pm 0.012 | No clear effect |
| Add fixed inputs to the PT-TNP decoder | PT-TNP (T) · ERA5 2-D Europe 2019 | test log-likelihood: 1.06 \pm 0.01 -> 1.22 \pm 0.01 | Helped |
| Add fixed inputs and zero them outside the training domain | PT-TNP (T) · ERA5 2-D US 2019 | test log-likelihood: 0.55 \pm 0.01 -> 0.55 \pm 0.01 | No clear effect |
| Add fixed inputs to the PT-TNP decoder | PT-TNP (T) · ERA5 4-D Europe 2019 | test log-likelihood: 1.14 \pm 0.01 -> 1.21 \pm 0.01 | Helped |
| Add fixed inputs to the PT-TNP decoder | PT-TNP (T) · ERA5 4-D US 2019 | test log-likelihood: 0.73 \pm 0.01 -> 0.76 \pm 0.01 | Helped |
| Increase fixed-input basis functions from 0 to 1 (ablation) | Approximately equivariant NPs · Synthetic 1-D GP | test log-likelihood | Helped |
Surprising findings
- Approximately equivariant models outperform equivariant counterparts even on out-of-distribution US data where fixed inputs are zeroed out, suggesting the approach regularises the equivariant component.
- Using more fixed inputs than necessary does not hurt performance in the experiments.
Limitations
- Degree of approximate equivariance is not rigorously quantified or controlled.
- Only simple approaches to incorporating approximate equivariance were considered.
- Experiments are relatively small-scale.
Experimental setup
Datasets: Gibbs-kernel GP synthetic 1-D regression 2-D smoke plume simulations (PhiFlow) ERA5 surface air temperature 2018-2019
Benchmarks: 1-D GP Gibbs kernel (ID/OOD) 2-D smoke plumes ERA5 2-D and 4-D environmental regression (Europe, central US)
Metrics: average test log-likelihood
Baselines: TNP ConvCNP RelaxedConvCNP EquivCNP PT-TNP
Builds on: Neural processes ConvCNP EquivCNP TE-TNP Relaxed group convolutions
Closest papers by meaning
- Flow Equivariant Recurrent Neural Networks (NeurIPS 2025 · similarity 0.90)
- Learning (Approximately) Equivariant Networks via Constrained Optimization (NeurIPS 2025 · similarity 0.90)
- Translation Equivariant Transformer Neural Processes (ICML 2024 · similarity 0.89)
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.89)
- Improving Equivariant Networks with Probabilistic Symmetry Breaking (ICLR 2025 · similarity 0.89)
- Equivariant Neural Tangent Kernels (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.