Research map / Optimization / Implicit bias and generalization
On the Benefits of Active Data Collection in Operator Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that for linear operators with inputs drawn from mean-zero processes with continuous covariance kernels, an active data collection strategy (querying Karhunen–Loève eigenfunctions) yields error bounds decaying with covariance eigenvalues (arbitrarily fast rates), while any passive i.i.d. strategy has a non-vanishing lower bound.
Problem
Sample complexity of operator learning for linear solution operators (e.g., PDEs) when input functions can be chosen actively rather than drawn i.i.d., under a stochastic-process input distribution with a continuous covariance kernel.
Approach
Use the Karhunen–Loève eigenfunctions of the input covariance kernel as the actively chosen source terms and a simple orthonormal-projection estimator; the excess risk is then bounded by the tail sum of covariance eigenvalues, giving rates set by eigenvalue decay, while passive i.i.d. sampling admits a non-vanishing minimax lower bound.
Key results
- Active linear estimator achieves error bound ε² Σ_{i≤n} λ_i + ||F||²_op Σ_{i>n} λ_i; for covariance α(-∇²+βI)^{-γ} the reducible error decays like n^{-(2γ/d-1)}.
- Any passive (i.i.d.) strategy has a minimax lower bound of (||F||²_op/2) Σ_{j≤m} λ_j for every m, so the risk does not vanish with n (e.g., ≥ ||F||²_op λ_1/2 for m=1).
- Experiments on Poisson and heat equations show the active linear estimator's error decreases faster than passive linear and FNO estimators as n grows.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use active eigenfunction sampling instead of passive i.i.d. sampling | Linear operator learning · General continuous covariance kernel | expected squared L2 error | Helped |
| Use active data collection instead of passive random sampling | Linear estimator for the Poisson equation · Poisson Equation | mean-squared relative error | Helped |
| Use active data collection instead of passive random sampling | Operator learning for the Heat equation · Heat Equation | relative testing error | Helped |
| Use active data collection with FNO instead of passive data collection with FNO | Fourier Neural Operator · Poisson Equation | mean-squared relative error | Hurt |
Surprising findings
- Passive i.i.d. data admits a non-vanishing lower bound regardless of covariance eigenvalue decay, even though passive rates are usually assumed to be ~1/n.
- FNO trained on actively collected data performed poorly, so it was excluded from the main comparison.
Limitations
- Results cover only bounded linear operators; nonlinear operators are left to future work.
- Guarantees hold in expectation over the input distribution, weaker than uniform L2 guarantees in some related work.
- Requires the learner to know the covariance kernel and to query the oracle on arbitrary (eigenfunction) inputs (membership query model), which may be unrealistic for some settings.
Experimental setup
Datasets: synthetic Gaussian process source functions for Poisson equation synthetic Gaussian process initial conditions for heat equation
Benchmarks: Poisson equation (2D, Dirichlet) Heat equation (2D)
Metrics: mean-squared relative L2 error
Baselines: Fourier Neural Operator least-squares linear estimator on passive data
Builds on: Karhunen-Loeve theorem membership query model Fourier Neural Operator
Closest papers by meaning
- Improved Active Learning via Dependent Leverage Score Sampling (ICLR 2024 · similarity 0.86)
- Active Learning as Nullspace Regulation: A Spectral Representation Perspective (NeurIPS 2026 · similarity 0.86)
- Constants Matter: The Performance Gains of Active Learning (ICML 2022 · similarity 0.86)
- A Unified Framework for Learning with Nonlinear Model Classes from Arbitrary Linear Samples (ICML 2024 · similarity 0.84)
- On the Convergence of Loss and Uncertainty-based Active Learning Algorithms (NeurIPS 2024 · similarity 0.84)
- Transductive Active Learning: Theory and Applications (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.