Research map / Learning theory and algorithms / Optimal transport
Automatic Outlier Rectification via Optimal Transport
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an automatic outlier rectification estimator that jointly rectifies contaminated data via an optimal transport ball with a concave cost, solving a min-min problem, and evaluates on mean estimation, LAD regression, and option implied volatility surfaces.
Problem
Outlier removal in estimation is typically two-stage and not informed by the downstream estimation task, leading to overly conservative removal or model-specific overfitting.
Approach
Builds a rectification set as an optimal transport ball (with concave cost r in (0,1)) around the contaminated empirical distribution and minimizes the expected loss over the best rectified distribution in that set, so the concave cost induces "long haul" moves that relocate only outliers; the joint min-min problem reduces to a finite convex program with quantile-like structure.
Key results
- Kernelized IVS: mean MAPE 0.225 for the proposed estimator vs 0.294 for KS and 0.236 for 2SKS; mean surface gradient 6.5 vs 20.2 (KS) and 7.5 (2SKS).
- Deep IVS (soft constraints): test RMSE and MAPE reduced by 33% and 34% relative to Dupire NN.
- Mean estimation and LAD regression: proposed estimator outperforms mean, median, trimmed mean, OLS, LAD and Huber in simulations, averaged over 100 trials.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use joint outlier rectification and estimation instead of KS | Kernelized IVS estimation · Kernelized IVS | MAPE (mean): 0.294 -> 0.225 | Helped |
| Use joint outlier rectification and estimation instead of 2SKS | Kernelized IVS estimation · Kernelized IVS | MAPE (mean): 0.236 -> 0.225 | Helped |
| Use joint outlier rectification and estimation instead of KS | Kernelized IVS estimation · Kernelized IVS | ∇Ŝ (mean): 20.188 -> 6.502 | Helped |
| Use joint outlier rectification and estimation instead of 2SKS | Kernelized IVS estimation · Kernelized IVS | ∇Ŝ (mean): 7.491 -> 6.502 | Helped |
| Choose δ by cross-validation instead of fixing δ=10⁻² (ablation) | Ours for kernelized IVS · Kernelized IVS | ∇Ŝ (mean): 6.502 -> 5.079 | Helped |
| Apply robust rectification to Dupire NN | Dupire NN, soft constraints · DAX IVS | RMSE: 0.044 -> 0.029; 33% reduction | Helped |
| Apply robust rectification to Dupire NN | Dupire NN, soft constraints · DAX IVS | MAPE: 0.111 -> 0.074; 34% reduction | Helped |
| Apply robust rectification to Dupire NN | Dupire NN · DAX IVS, hard constraints | RMSE: 0.125 -> 0.110 | Helped |
| Apply robust rectification to Dupire NN | Dupire NN · DAX IVS, hybrid constraints | RMSE: 0.140 -> 0.131 | Helped |
| Apply robust rectification to Dupire NN | Dupire NN · DAX IVS, ε=40% corruption | RMSE: 0.168 -> 0.141 | Helped |
| Apply robust rectification to Dupire NN | Dupire NN · DAX IVS, ε=40% corruption | MAPE: 0.115 -> 0.097 | Helped |
Surprising findings
- With a convex cost (r=2), all points, clean and corrupted, are rectified toward each other, yielding an incorrect line of best fit regardless of budget.
- The proposed estimator reaches its results even though the inner problem for general (non-LAD) losses is solved heuristically without convergence guarantees.
Limitations
- Theoretical reformulation holds for mean estimation and LAD regression; general losses use a heuristic without guarantees.
- Less suited to classification, where the OT-based rectification may be limited; phi-divergence alternative only discussed.
- Budget delta requires tuning via cross-validation.
Experimental setup
Datasets: WRDS US stock option chains 2019-2021 DAX index option chains (from prior work) synthetic Gaussian mixture data
Benchmarks: WRDS US option chains DAX option chains with contamination
Metrics: MAPE discretized surface gradient (smoothness) RMSE estimator error with confidence intervals
Baselines: mean median trimmed mean OLS LAD regression Huber regression KS (kernel smoothing) 2SKS (remove and fit kernel smoothing) Dupire NN
Builds on: Wasserstein distributionally robust optimization strong duality minimum distance functionals estimator Tukey fences kernel smoothing
Closest papers by meaning
- Outlier-Robust Distributionally Robust Optimization via Unbalanced Optimal Transport (NeurIPS 2024 · similarity 0.86)
- Tight Rates in Supervised Outlier Transfer Learning (ICLR 2024 · similarity 0.86)
- Beyond Empirical Support: Structured Outlier Generation via Sinkhorn Optimal Transport (NeurIPS 2026 · similarity 0.85)
- ALTBI: Constructing Improved Outlier Detection Models via Optimization of Inlier-Memorization Effect (AAAI 2025 · similarity 0.84)
- Estimation of Stochastic Optimal Transport Maps (NeurIPS 2025 · similarity 0.84)
- Density Ratio Estimation with Doubly Strong Robustness (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.