Research map / Learning theory and algorithms / Prediction and decision losses
REGLO: Provable Neural Network Repair for Global Robustness Properties
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes REGLO, a post-hoc provable repair method that finds ReLU linear regions violating a global robustness property via a gradient-based theorem and fixes them by a minimal last-layer weight change solved as robust convex optimization.
Problem
Repairing pretrained neural networks so they provably satisfy global robustness properties (and individual fairness as a special case), which training-based methods cannot guarantee and which existing DNN repair methods do not handle.
Approach
Shows any global robustness counterexample implies a point with large gradient, so violating ReLU linear regions can be found via MILP; then repairs them by minimal last-hidden-layer weight change formulated as a convex robust optimization problem, iterated until no violating regions remain.
Key results
- On German Credit individual fairness repair, REGLO reduces VB, PGD-B and PGD-R by up to 99.3% with a 0.9% accuracy drop.
- On GTSRB, ST+REGLO reduces PGD-R of ST from 100% to 0.0% and VB/PGD-B by up to 99%, keeping accuracy above 90% in most settings.
- Compared with VGRP on three datasets, REGLO is up to 99.86% faster in running time and achieves up to 0.88% higher accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply REGLO after standard training | German Credit classifier · German Credit, All age | VB: 12.5 -> 0.29 | Helped |
| Apply REGLO after standard training | German Credit classifier · German Credit, All age | PGD-B: 1.31 -> 0.028 | Helped |
| Apply REGLO after standard training | German Credit classifier · German Credit, All age | ACC: 76.6% -> 75.6% | Hurt |
| Apply REGLO after standard training | German Credit classifier · German Credit, All age | PGD-R: 75.5% -> 0.0% | Helped |
| Apply REGLO after standard training | German Credit classifier · German Credit, All age | PGD-B: 5.6 -> 0.2 | Helped |
| Apply REGLO after standard training | German Credit classifier · GTSRB classifier | PGD-B: 8.3 -> 0.08 | Helped |
| Apply REGLO after standard training | German Credit classifier · GTSRB classifier | PGD-B: 6.4 -> 0.1 | Helped |
| Apply REGLO after standard training | REGLO classifier · Cryptojacking | Acc: 98.10 -> 98.19 | Helped |
| Apply REGLO instead of VGRP | classifier trained on Twitter Spam Accounts · Twitter Spam Accounts | Acc: 92.20 -> 93.08 | Helped |
| Apply REGLO instead of VGRP | classifier trained on Twitter Spam URLs · Twitter Spam URLs | Acc: 96.50 -> 98.60 | Helped |
Surprising findings
- AT+REGLO yields larger robustness improvements but lower accuracy than ST+REGLO, consistent with the robustness-accuracy trade-off.
Limitations
- Relies on ITNE verification and MILP, which only scale to moderate-size networks.
- Number of linear regions grows exponentially with depth, so large DNNs need random sampling and lose deterministic guarantees.
- Weight modification in the last hidden layer may hurt performance.
Experimental setup
Datasets: German Credit GTSRB Cryptojacking Twitter Spam Accounts Twitter Spam URLs
Benchmarks: German Credit GTSRB Cryptojacking Twitter Spam Accounts Twitter Spam URLs
Metrics: verification bound (VB) PGD-B PGD-R accuracy running time
Baselines: ST AT ST+Finetune AT+Finetune CertiFair VGRP
Builds on: ITNE CertiFair VGRP booster-fixer PGD adversarial training Fu and Li 2021 sub-network repair
Closest papers by meaning
- Towards Fast Computation of Certified Robustness for ReLU Networks (ICML 2018 · similarity 0.87)
- Gradient Methods Provably Converge to Non-Robust Networks (NeurIPS 2022 · similarity 0.87)
- Probably Approximately Global Robustness Certification (ICML 2025 · similarity 0.86)
- Convex Formulations for Training Two-Layer ReLU Neural Networks (ICLR 2025 · similarity 0.85)
- Enhancing Robust Fairness via Confusional Spectral Regularization (ICLR 2025 · similarity 0.84)
- Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.