Research map / Probabilistic and scientific modeling / Differential equations and PDEs

In-Context Multi-Operator Learning with DeepOSets

NeurIPS 2026 · Probabilistic and scientific modeling › Differential equations and PDEs · approach: Neural operators for general geometries · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Formalizes in-context multi-operator learning over compact operator classes and proves uniform universality of a DeepOSets (DeepSets encoder + DeepONet decoder) architecture, with experiments on 6 and 12 IVP/BVP tasks trained in minutes.

Problem

Multi-operator (in-context) learning of PDE solution operators lacks a precise mathematical formulation and universality guarantees; existing ICON-style transformers are quadratic in prompt length and slow to train.

Approach

Use a DeepSets encoder to pool embeddings of prompt input-output pairs into a permutation-invariant latent vector, fed with the query function into a DeepONet branch net, and prove this architecture uniformly universal over a compact set of operators under a (delta,C)-discretization of the prompt inputs.

Key results

• The 12-task DeepOSets reaches test MSE on the order of 1e-4 to 1e-2 across tasks (e.g., 4.87e-04 on forward Poisson BVP), with errors mostly lower than the 6-task model.
• Training the 12-task DeepOSets took about 38 minutes on a single Nvidia 4060 Ti GPU, versus hours for the transformer-based baseline cited.
• Trained with context size N=4, the model's MSE decreases as the inference-time context grows from 3 to 20 examples.

Surprising findings

• Training on the 12-task mixture (IVP plus BVP) gave lower errors than training on the 6 BVP tasks alone, which the authors attribute to greater solution diversity.

Limitations

• Experiments are limited to 1D ODE/PDE problems on a fixed uniform 100-point grid.
• Baseline comparison to transformer-based ICON is only stated via reported training times, not a direct accuracy comparison.
• Theorem is existential and does not give explicit network sizes or sample complexity.

Experimental setup

Datasets: synthetic Gaussian-process parameter functions with numerically solved ODE/PDE solutions

Benchmarks: 12 forward/inverse IVP and BVP tasks (synthetic)

Metrics: mean squared error

Baselines: ICON

Builds on: DeepSets DeepONet In-Context Operator Networks (ICON) PROSE

Closest papers by meaning

• Derivative-enhanced Deep Operator Network (NeurIPS 2024 · similarity 0.88)
• Optimal deep learning of holomorphic operators between Banach spaces (NeurIPS 2024 · similarity 0.88)
• Operator-Learning-Inspired Modeling of Neural Ordinary Differential Equations (AAAI 2024 · similarity 0.87)
• MgNO: Efficient Parameterization of Linear Operators via Multigrid (ICLR 2024 · similarity 0.87)
• Smooth Piecewise Cutting for Neural Operator to Handle Discontinuities and Sharp Transitions (NeurIPS 2026 · similarity 0.87)
• Neural Operators with Localized Integral and Differential Kernels (ICML 2024 · similarity 0.87)