Research map / Optimization / Optimizers
Deep Legendre Transform
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Deep Legendre Transform (DLT), which trains a neural network on the implicit Fenchel identity f*(∇f(x)) = <x,∇f(x)> - f(x) to approximate convex conjugates in high dimensions, with a posteriori L2 error estimates and KAN symbolic regression for exact forms.
Problem
Computing convex conjugates (Legendre–Fenchel transforms) of differentiable convex functions is intractable in high dimensions for grid-based methods, and prior neural approaches target optimal transport or require complex max-min optimization.
Approach
Replace the unknown conjugate target with the implicit Fenchel–Young identity so that a regressor g(∇f(x)) is trained on exact targets <x,∇f(x)> - f(x) at sampled primal points; this gives unbiased a posteriori L2 error estimates and, with a learned inverse-gradient map for sampling, controls the dual-space distribution. KANs are used for symbolic regression to recover exact conjugates.
Key results
- DLT matches direct supervised learning of known conjugates (e.g., ResNet quadratic d=10 RMSE 2.43e-5 DLT vs 3.00e-5 direct).
- Versus Lucet's grid method with N=10, DLT ResNet achieves RMSE 1.33e-1 at d=8 where Lucet gives 2.93e+1, and runs in 0.02 s evaluation time, scaling to d=10.
- KAN symbolic regression recovers exact conjugates in 2D, e.g., sum x_i^2 gives 0.25 y_1^2 + 0.25 y_2^2 with L2 error 0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DLT instead of direct supervised learning of the known conjugate | ResNet convex-conjugate approximation · Quadratic, d=10 | RMSE: 2.43e-5 vs 3.00e-5 | No clear effect |
| Use DLT instead of direct supervised learning of the known conjugate | ResNet convex-conjugate approximation · Neg. Log, d=10 | RMSE: 7.34e-4 vs 9.22e-4 | No clear effect |
| Use DLT instead of direct supervised learning of the known conjugate | ResNet convex-conjugate approximation · Neg. Entropy, d=10 | RMSE: 6.62e-4 vs 7.41e-4 | No clear effect |
| Use ResNet rather than Lucet's grid method | Convex-conjugate approximation · Neg. Log, d=8 | RMSE: 2.93e$+$01 -> 1.33e$-$01 | Helped |
| Use inverse sampling from the dual space rather than direct sampling from the primal space | DLT for the negative logarithm · Neg. Log, d=10 | L2 approximation error: 3.18e+00 -> 4.07e-02 | Helped |
| Use Time-DLT rather than DGM | Hamilton–Jacobi solution, d=10 · t=0.1 | L2 Error: 8.90e-2 -> 8.46e-2 | Helped |
| Use Time-DLT rather than DGM | Hamilton–Jacobi solution, d=10 · t=1.0 | L2 Error: 2.26e-1 -> 3.68e-2 | Helped |
| Use ResNet instead of ICNN for DLT approximation (ablation) | DLT · Multidimensional conjugate approximation | approximation error | Helped |
| Add skip connections by using ICNN rather than MLP-ICNN (ablation) | DLT convex-conjugate approximation · Quadratic, Neg. Log, and Neg. Entropy | RMSE | Mixed |
| Use approximate inverse sampling instead of direct primal sampling (ablation) | DLT for the negative logarithm · Neg. Log, d=2–100 | L2 approximation error: 3.18e-02 -> 5.15e-03 (d=2); 5.44e+01 -> 3.06e+00 (d=100) | Helped |
| Increase the softmax smoothing parameter ε (ablation) | Peyré quasi-Monte Carlo approximation · Neg. Log, d=2 | RMSE: 6.54e$-$02 -> 5.20e$-$01 | Hurt |
Surprising findings
- Inverse (learned generalized-inverse) sampling gives 1-2 orders of magnitude lower error for negative log at d≥10 but increases training time substantially.
- Direct DGM outperforms Time-DLT near t=0, while Time-DLT wins at larger t; the Time-DLT comparison uses the explicit g* so it is not a fully fair comparison.
Limitations
- Requires the function f to be differentiable and convex on an open convex set.
- Approximates f* only on the gradient image D = ∇f(C), not the full domain of f*.
- Training time grows with more complex non-separable functions and inverse sampling adds cost.
Experimental setup
Benchmarks: analytic convex conjugates (quadratic, negative log, negative entropy, quadratic-over-linear) Hamilton-Jacobi quadratic Hopf formula test
Metrics: L2 approximation error (RMSE) training time PDE residual
Baselines: Linear-time Legendre Transform (Lucet) Deep Galerkin Method direct learning of conjugate softmax/Gaussian convolution approach (Peyré)
Builds on: input convex neural networks kolmogorov-arnold networks deep galerkin method linear-time legendre transform
Closest papers by meaning
- A Deep Conjugate Direction Method for Iteratively Solving Linear Systems (ICML 2023 · similarity 0.88)
- SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions (NeurIPS 2026 · similarity 0.85)
- Bridging Discrete and Backpropagation: Straight-Through and Beyond (NeurIPS 2023 · similarity 0.84)
- Solving hidden monotone variational inequalities with surrogate losses (ICLR 2025 · similarity 0.84)
- BPQP: A Differentiable Convex Optimization Framework for Efficient End-to-End Learning (NeurIPS 2024 · similarity 0.84)
- How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.