Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Dimension Agnostic Neural Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Dimension Agnostic Neural Processes (DANP), which adds a dimension aggregator block and a Transformer latent path to TNP so one model handles varying input/output dimensions in regression.
Problem
Neural Processes are typically tied to fixed input and output dimensions, so a separate model must be trained for each dimensionality, limiting their use as general regressors.
Approach
A Dimension Aggregator Block maps each input dimension to a shared embedding space with positional encodings and self-attention, then average-pools x into a fixed-size vector while keeping y per dimension; a Transformer-based latent path is added to the TNP deterministic path.
Key results
- On 1D RBF GP from-scratch, DANP reaches 0.921 target log-likelihood versus 0.904 for TNP and 0.855 for ANP.
- In zero-shot GP regression with DANP trained on 2D, 3D and 4D, 1D RBF target log-likelihood is 0.826 without any 1D training.
- In fine-tuning on CelebA video, DANP achieves 0.560 target log-likelihood while all baselines are negative (TNP -2.747).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DANP instead of TNP | Neural Process regression · 1D and 2D GP regression, from-scratch target | log-likelihood: 0.904 -> 0.921; 0.710 -> 0.723; 0.362 -> 0.373; 0.060 -> 0.068 | Helped |
| Use DANP instead of TNP | Neural Process image completion · EMNIST | target log-likelihood: 0.945 ± 0.004 -> 0.969 ± 0.002 | Helped |
| Use DANP instead of the strongest baseline, BANP | Neural Process image completion · CelebA | target log-likelihood: 2.292 ± 0.021 -> 2.027 ± 0.006 | Hurt |
| Add DAB to TNP (ablation) | TNP · 2D RBF GP, EMNIST, and CelebA targets | log-likelihood: 0.362 -> 0.365; 0.945 -> 0.949; 1.632 -> 1.645 | Helped |
| Add a latent path to TNP (ablation) | TNP · 2D RBF GP, EMNIST, and CelebA targets | log-likelihood: 0.362 -> 0.371; 0.945 -> 0.967; 1.632 -> 1.973 | Helped |
| Add positional encoding (ablation) | DANP · 2D RBF GP, EMNIST, and CelebA targets | log-likelihood: -0.395 -> 0.373; 0.376 -> 0.969; 0.631 -> 2.027 | Helped |
| Replace mean pooling with PMA (ablation) | DANP · 1D and 2D GP regression, EMNIST, and CelebA targets | log-likelihood: 0.921 -> 0.921; 0.723 -> 0.721; 0.373 -> 0.372; 0.068 -> 0.067; 0.969 -> 0.975; 2.027 -> 2.025 | No clear effect |
Surprising findings
- NDP shows much higher context log-likelihood (about 5.9) but strongly negative target log-likelihood (about -0.4 to -0.7 in 1D and 2D), so it underperforms DANP on target points.
- Adding the DAB module alone to TNP gives performance close to TNP, while the latent path gives most of the gain.
Limitations
- Evaluated only on regression tasks; extension to classification is left as future work.
- Target log-likelihood gains over TNP are modest on several GP settings (for example 0.923 vs 0.904 in 1D RBF with latent path).
Experimental setup
Datasets: synthetic GP regression data EMNIST CelebA CelebA video (constructed) HPO-B CIFAR-10
Benchmarks: GP regression (1D/2D RBF and Matern, zero-shot, fine-tuning) EMNIST image completion CelebA image completion CelebA video fine-tuning HPO-B synthetic BO benchmarks (Ackley, Cosine, Rastrigin)
Metrics: context log-likelihood target log-likelihood normalized regret CRPS confidence interval coverage
Baselines: CANP ANP BANP MPANP TNP NDP
Builds on: Transformer Neural Processes Neural Process Attentive Neural Process Bootstrapping Attentive Neural Process Martingale Posterior Neural Process
Closest papers by meaning
- Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling (ICML 2022 · similarity 0.87)
- Accurate Bayesian Meta-Learning by Accurate Task Posterior Inference (ICLR 2023 · similarity 0.86)
- Learning to Generalize: An Information Perspective on Neural Processes (NeurIPS 2025 · similarity 0.85)
- Flow Matching Neural Processes (NeurIPS 2025 · similarity 0.84)
- Spectral Convolutional Conditional Neural Process (NeurIPS 2025 · similarity 0.83)
- Gaussian Process Neural Additive Models (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.