Research map / Learning theory and algorithms / Prediction and decision losses
Attentive Neural Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add query-specific cross-attention over context points | Neural Process · 1D GP regression, random kernel hyperparameters | context reconstruction error | Helped |
| Add query-specific cross-attention over context points | Neural Process · 1D GP regression, random kernel hyperparameters | target NLL | Helped |
| Use learned dot-product attention instead of fixed Laplace attention (ablation) | Attentive Neural Process · 1D GP regression | context reconstruction | Helped |
| Use dot-product attention instead of multihead attention (ablation) | Attentive Neural Process · 1D GP regression | prediction smoothness | Hurt |
| Use multiple attention heads instead of a single dot-product attention (ablation) | Attentive Neural Process · 1D GP regression | context reconstruction | Helped |
| Increase the Neural Process bottleneck size (ablation) | Neural Process · 1D GP regression | context reconstruction error | Mixed |
| Replace a larger Neural Process bottleneck with multihead attention (ablation) | Neural Process · 1D GP regression | context reconstruction error | Helped |
| Add multihead cross-attention | Neural Process · 2D image regression | context reconstruction error | Helped |
| Add multihead cross-attention | Neural Process · 2D image regression | target NLL | Helped |
| Add two stacked self-attention layers in the deterministic and latent paths (ablation) | Multihead ANP · 2D image regression | target NLL | Helped |
| Add two stacked self-attention layers in the deterministic and latent paths (ablation) | Multihead ANP · 2D image regression | image crispness and global coherence | Helped |
| Stack more than two self-attention layers (ablation) | Stacked Multihead ANP · 2D image regression | qualitative and quantitative performance | No clear effect |
Closest papers by meaning
- Residual Neural Processes (AAAI 2020 · similarity 0.89)
- Rényi Neural Processes (ICML 2025 · similarity 0.86)
- Flow Matching Neural Processes (NeurIPS 2025 · similarity 0.86)
- Memory Efficient Neural Processes via Constant Memory Attention Block (ICML 2024 · similarity 0.85)
- In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness (NeurIPS 2024 · similarity 0.84)
- How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression? (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.