Research map / Learning theory and algorithms / Combinatorial and search optimization

SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions

NeurIPS 2026 · Learning theory and algorithms › Combinatorial and search optimization · approach: Semidefinite relaxations for verification · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SOC-ICNN, which extends ReLU-ICNNs (LP value functions) with quadratic and Euclidean-norm conic branches so the forward pass equals an SOCP value function, and reports better approximation on ten convex targets and mixed downstream regret.

Problem

ReLU-based input convex neural networks represent only piecewise-linear convex functions, which is parameter-inefficient for smooth curvature and non-polyhedral geometry such as Euclidean norms.

Approach

Augment the ReLU-ICNN LP value-function backbone with PSD quadratic branches and norm (conic) branches with nonnegative weights, so the forward pass is exactly the optimal value of an SOCP while keeping feed-forward inference cost.

Key results

• On ten convex targets at d=50, SOC-ICNN achieves mean relative error 0.099 versus 0.772 for ReLU-ICNN and 0.763 for Softplus-ICNN, with fewer parameters (9,423).
• Value-function equivalence experiment: primal-dual gap about 1e-14 and forward-vs-CVXPY error about 5.6e-7 over 150 random trials.
• On six downstream tasks at d=10, PCF has the lowest mean regret (0.139), while SOC-ICNN (0.197) is competitive but not best; Quad-ICNN reaches 0.179.

Surprising findings

• The full SOC-ICNN is not uniformly best downstream: PCF and Quad-ICNN outperform it on regret, and the Norm branch is described as unstable in the decision setting.
• Pointwise approximation gains do not translate uniformly into better decision quality, as the paper notes a gap between fitting accuracy and decision regret.

Limitations

• Theoretical guarantees are limited to finite-dimensional and finite-horizon structured extensions; implicit or infinite-horizon recurrent models need extra stability assumptions.
• Downstream decision quality is not uniformly improved; PCF remains best on average regret.
• Experiments use synthetic convex targets and six parametric tasks, with only three seeds in the approximation study.

Experimental setup

Datasets: synthetic convex target functions (ten targets, d in {5,10,20,50}) six parametric convex optimization tasks

Benchmarks: ten synthetic convex targets (relative l2 error) six parametric convex optimization tasks (regret)

Metrics: relative l2 error regret parameter count primal-dual gap forward-solver error

Baselines: ReLU-ICNN Softplus-ICNN Quad-ICNN Norm-ICNN DCP PCF

Builds on: Input Convex Neural Networks ICKAN OptNet Learning Parametric Convex Functions (PCF) differentiable convex optimization layers

Closest papers by meaning

• On the Universality and Complexity of GNN for Solving Second-order Cone Programs (ICLR 2026 · similarity 0.88)
• Convex Formulations for Training Two-Layer ReLU Neural Networks (ICLR 2025 · similarity 0.87)
• Expressivity of ReLU-Networks under Convex Relaxations (ICLR 2024 · similarity 0.85)
• Deep Legendre Transform (NeurIPS 2025 · similarity 0.85)
• On the Power of Small-size Graph Neural Networks for Linear Programming (NeurIPS 2024 · similarity 0.84)
• Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time (ICML 2024 · similarity 0.84)