Research map / Label-efficient and robust learning / Recognition and long-tail
Curvature Enhanced Data Augmentation for Regression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CEMS, a second-order manifold-based data augmentation method for regression that samples from a curvature-aware local approximation of the input-output data manifold.
Problem
Data augmentation for regression is less developed than for classification; first-order manifold-based augmentation (FOMA) fails to capture curved data structure, causing off-manifold samples.
Approach
Extends FOMA's tangent-space sampling to a second-order (Hessian-including) local parameterization of the joint input-output manifold, estimated via least squares on SVD-based local coordinates, and samples new points in the tangent space that are mapped back through the learned embedding.
Key results
- CEMS achieves best or second-best error on all four in-distribution datasets, e.g., Airfoil RMSE 1.455 vs. FOMA 1.471 and NO2 RMSE 0.507 vs. 0.509 for C-Mixup.
- On out-of-distribution benchmarks CEMS achieves the best result in 6 of 9 tests, e.g., SkillCraft average RMSE 5.142 and worst 6.322 versus the next best 5.201 and 6.877.
- CEMS training time is on par with competing methods (e.g., Airfoil 12.2 s vs. 11.64 s for C-Mixup and 7.11 s for FOMA).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace first-order FOMA sampling with second-order curvature-aware CEMS | Regression data augmentation · Airfoil | RMSE: 1.471 -> 1.455 | Helped |
| replace first-order FOMA sampling with second-order curvature-aware CEMS | Regression data augmentation · NO2 | RMSE: 0.512 -> 0.507 | Helped |
| replace first-order FOMA sampling with second-order curvature-aware CEMS | Regression data augmentation · Exchange-Rate | RMSE: 0.013 -> 0.014 | Hurt |
| replace first-order FOMA sampling with second-order curvature-aware CEMS | Regression data augmentation · SkillCraft | Avg. RMSE: 5.201 -> 5.142 | Helped |
| replace first-order FOMA sampling with second-order curvature-aware CEMS | Regression data augmentation · DTI | Avg. R: 0.503 -> 0.511 | Helped |
| replace first-order FOMA sampling with second-order curvature-aware CEMS | Regression data augmentation · Poverty | Worst R: 0.53 -> 0.50 | Hurt |
| use probability-weighted k-nearest-neighbor batches instead of k-nearest-neighbor batches (ablation) | CEMS · SkillCraft | Avg. RMSE: 5.142 -> 4.941 | Helped |
| use probability-weighted k-nearest-neighbor batches instead of k-nearest-neighbor batches (ablation) | CEMS · RCF | RMSE: 0.146 -> 0.173 | Hurt |
| use random batches instead of k-nearest-neighbor batches (ablation) | CEMS · Airfoil | RMSE: 1.455 -> 1.435 | Helped |
| use random batches instead of k-nearest-neighbor batches (ablation) | CEMS · DTI | R: 0.511 -> 0.491 | Hurt |
| compute and reuse one basis per neighborhood instead of computing a basis per point (ablation) | CEMS · Airfoil | RMSE: 1.462 -> 1.455 | No clear effect |
| compute and reuse one basis per neighborhood instead of computing a basis per point (ablation) | CEMS · NO2 | RMSE: 0.503 -> 0.507 | No clear effect |
Surprising findings
- On Exchange-Rate and Electricity, CEMS is only comparable to FOMA rather than clearly better.
- CEMS_p (per-point basis) does not consistently outperform the shared-neighborhood CEMS in the ablation.
Limitations
- Linear system may be underdetermined for large intrinsic dimension d, requiring many neighbors or ridge regularization.
- Full SVD may be needed when batch size is below intrinsic dimension, costing O(bD^2) memory for high-dimensional data.
Experimental setup
Datasets: Airfoil Self-Noise NO2 Exchange-Rate Electricity RCFashionMNIST Communities and Crime SkillCraft1 Master Table Drug-Target Interactions PovertyMap
Benchmarks: Airfoil Self-Noise NO2 Exchange-Rate Electricity RCFashionMNIST Communities and Crime SkillCraft1 Master Table Drug-Target Interactions PovertyMap
Metrics: RMSE MAPE R (Pearson correlation)
Baselines: ERM Mixup Manifold Mixup C-Mixup Anchor Data Augmentation (ADA) FOMA
Builds on: FOMA curvature-aware manifold learning (CAML) C-Mixup Manifold Mixup
Closest papers by meaning
- The Data Manifold under the Microscope (ICML 2026 · similarity 0.89)
- First-Order Manifold Data Augmentation for Regression Learning (ICML 2024 · similarity 0.89)
- Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive (NeurIPS 2025 · similarity 0.85)
- Diffusion Models Encode the Intrinsic Dimension of Data Manifolds (ICML 2024 · similarity 0.85)
- Recovering Manifold Structure Using Ollivier Ricci Curvature (ICLR 2025 · similarity 0.85)
- Hardness of Learning Neural Networks under the Manifold Hypothesis (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.