Research map / Learning theory and algorithms / Combinatorial and search optimization
Enforcing Hard Linear Constraints in Deep Learning Models with Decision Rules
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a model-agnostic architecture that enforces input-dependent linear equality and inequality constraints on neural network outputs by convexly combining a task network with a linear-decision-rule safe network, solved offline via SDP/LP.
Problem
Deep learning models need to satisfy hard input-dependent linear equality and inequality constraints (physical, fairness, safety) at training and inference time, while existing penalty or projection methods are soft, slow, or limited to simple constraints.
Approach
A task network (trained for accuracy, equality-projected in closed form) is convexly combined with a safe network built from linear decision rules (robust-optimization style) whose outputs are feasible for all inputs in the uncertainty set; the blending weight alpha(x) is computed in closed form from slack ratios so the final output is feasible in a single forward pass.
Key results
- On portfolio optimization, the proposed method achieves a 2.35% mean optimality gap with zero constraint violation and 2.8 ms inference, versus 29.6% for DC3 and 48.56% for LDR.
- On the 200-bus QP DC-OPF, the proposed method achieves a 1.07% optimality gap with zero violations at 2.6 ms, versus 225 ms for APM.
- Inference times stay under 3 ms across tasks, faster than Gurobi and OSQP solvers on large cases, though it is less accurate than EAPM on some QP cases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine the task network with the feasible safe network using a convex combination | Decision-rule constrained network · Portfolio optimization | Inequality Violation: 0.053(0.123) -> 0.000(0.000) | Helped |
| combine the task network with the feasible safe network using a convex combination | Decision-rule constrained network · Portfolio optimization | Optimality Gap: 29.6(36.7) -> 2.35(6.37) | Helped |
| use the proposed non-iterative framework instead of APM | DC-OPF QP 118-bus · DC-OPF QP 118-bus | Inference Time (ms): 117.6(156) -> 2.5(1) | Helped |
| use the proposed framework instead of post-projection with Gurobi | DC-OPF QP 118-bus · DC-OPF QP 118-bus | Optimality Gap: 0.11(1.88) -> 1.10(2.13) | Hurt |
| use the proposed framework instead of post-projection with OSQP | DC-OPF QP 200-bus · DC-OPF QP 200-bus | Optimality Gap: 0.22(1.49) -> 1.07(1.71) | Hurt |
| add task-network pre-training on the safe network output (ablation) | Decision-rule constrained network · Portfolio optimization | Optimality Gap: 3.14(6.36) -> 2.35(6.37) | Helped |
| add task-network pre-training on the safe network output (ablation) | Decision-rule constrained network · DC-OPF 14-bus | Optimality Gap: 0.00(0.00) -> 0.00(0.00) | No clear effect |
| add task-network pre-training on the safe network output (ablation) | Decision-rule constrained network · DC-OPF 30-bus | Optimality Gap: 10.60(17.63) -> 0.00(0.00) | Helped |
| add task-network pre-training on the safe network output (ablation) | Decision-rule constrained network · DC-OPF 57-bus | Optimality Gap: 0.23(0.72) -> 0.21(0.68) | No clear effect |
| add task-network pre-training on the safe network output (ablation) | Decision-rule constrained network · DC-OPF 118-bus | Optimality Gap: 1.15(2.65) -> 1.27(2.00) | Mixed |
| add task-network pre-training on the safe network output (ablation) | Decision-rule constrained network · DC-OPF 200-bus | Optimality Gap: 0.89(1.35) -> 0.99(1.78) | Mixed |
Surprising findings
- On the LP DC-OPF 118-bus and 200-bus cases, the proposed method's optimality gap (1.27%, 0.99%) is larger than APM's (0.07%, 0.72%) and comparable to or worse than DC3 on some metrics, showing trade-offs.
- Pre-training the task network toward the safe network output substantially helped the 30-bus DC-OPF case (10.60% gap without vs 0.00% with).
Limitations
- Linear decision rules may limit expressiveness in highly nonlinear settings and increase conservativeness.
- Assumes polyhedral/convex input spaces and constraint sets; the SDP inner approximation is conservative when multiple quadratic constraints define the input set.
- Gradient through the max operator for alpha is sparse and potentially unstable.
Experimental setup
Datasets: DC-OPF test cases (14, 30, 57, 118, 200-bus) portfolio optimization bond data
Benchmarks: DC-OPF (QP) 14/30/57/118/200-bus DC-OPF (LP) 14/30/57/118/200-bus Portfolio optimization
Metrics: optimality gap equality violation inequality violation inference time (ms)
Baselines: Gurobi OSQP Post projection APM EAPM DC3 LDR
Builds on: gauge mapping DC3 decision rules S-lemma
Closest papers by meaning
- DeepSaDe: Learning Neural Networks That Guarantee Domain Constraint Satisfaction (AAAI 2024 · similarity 0.88)
- A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks (NeurIPS 2019 · similarity 0.88)
- DisjunctiveNet: Neural Symbolic Learning via Differentiable Convexified Optimization Layers (ICML 2026 · similarity 0.87)
- VNN: Verification-Friendly Neural Networks with Hard Robustness Guarantees (ICML 2024 · similarity 0.86)
- Towards General Robustness Verification of MaxPool-based Convolutional Neural Networks via Tightening Linear Approximation (CVPR 2024 · similarity 0.85)
- Understanding Certified Training with Interval Bound Propagation (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.