Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Incremental Transformer Neural Processes
Philip Mortimer, Cristiana Diaconu, Tommy Rochussen, Bruno Kacper Mlodozeniec, Richard E. Turner
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dense autoregressive sequence training (incTNP-Seq) to incTNP (ablation) | incTNP · Protein (factorised) | test log-likelihood delta vs TNP-D: -0.028 -> +0.036 | Improved |
| add dense autoregressive sequence training to incTNP (ablation) | incTNP · Temperature (Forecast) (factorised) | test log-likelihood delta vs TNP-D: +0.030 -> +0.690 | Improved |
| add dense autoregressive sequence training to incTNP (ablation) | incTNP · 1D GP (factorised) | test log-likelihood delta vs TNP-D: -0.013 -> -0.002 | Improved |
| add dense autoregressive sequence training to incTNP (ablation) | incTNP · Elevators (factorised) | test log-likelihood delta vs TNP-D: -0.024 -> -0.009 | Improved |
| replace TNP-D with causally masked incTNP-Seq in AR deployment | TNP-D · Temperature (Interp) (AR) | inference cost ratio: 3.57x -> 1.00x | Improved |
| replace TNP-D with incTNP-Seq in AR deployment | TNP-D · Temperature (Forecast) (AR) | test log-likelihood: -1.749 -> +0.061 delta | Improved |
| use incTNP-Seq instead of CNP | CNP · 1D GP (factorised) | test log-likelihood delta vs TNP-D: -0.230 -> -0.002 | Improved |
| use incTNP-Seq instead of LBANP | LBANP · Tabular (Synthetic) (factorised) | test log-likelihood delta vs TNP-D: -0.058 -> +0.007 | Improved |
| give TNP-D and incTNP equal compute budget to incTNP-Seq (ablation) | TNP-D and incTNP · 1D GP and Tabular hyperparameter sweep | peak test log-likelihood | No clear effect |
| add dense autoregressive training for hyperparameter robustness (ablation) | incTNP · 1D GP and Tabular sweep | variance across configurations | Improved |
| add causal masking to TNP-D (incTNP) for implicit Bayesianness | TNP-D · 1D GP | KL gap | No clear effect |
| use incTNP-Seq instead of WISKI streaming GP | WISKI · 1D GP streaming | KL gap and performance | Improved |
Closest papers by meaning
- Gridded Transformer Neural Processes for Spatio-Temporal Data (ICML 2025 · similarity 0.88)
- Translation Equivariant Transformer Neural Processes (ICML 2024 · similarity 0.86)
- Spectral Transformer Neural Processes (NeurIPS 2026 · similarity 0.85)
- Towards Understanding the Universality of Transformers for Next-Token Prediction (ICLR 2025 · similarity 0.84)
- Non-asymptotic Convergence of Training Transformers for Next-token Prediction (NeurIPS 2024 · similarity 0.84)
- TPP-SD: Accelerating Transformer Point Process Sampling with Speculative Decoding (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
