Research map / Architectures and efficiency / RNNs and normalization
VeriCompress: A Tool to Streamline the Synthesis of Verified Robust Compressed Neural Networks from Scratch
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces VeriCompress, a tool that searches for and trains parameter-budgeted sparse subnetworks of a randomly initialized backbone from scratch with verified robustness training (CROWN-IBP), outperforming Hydra and FaShapley on CIFAR-10, SVHN, MNIST and pedestrian detection.
Problem
Verified robust neural networks are large and costly to train, while safety-critical deployment on resource-constrained platforms needs compact models; existing robust pruning methods require pretraining a dense model and are slow and hyperparameter-heavy.
Approach
Adapts dynamic sparse training to verified robust training: a random sub-network under a parameter budget is trained with CROWN-IBP loss while dormant elements receive gradients, and the least important kernels/nodes (by l2-norm, layer-wise with Erdos-Renyi-Kernel scaling) are periodically deactivated, with no pretraining or fine-tuning phase.
Key results
- VeriCompress improves standard and verified accuracy over FaShapley/Hydra by an average of 15.1 and 9.8 percentage points across CIFAR-10 and SVHN settings.
- Training takes about 3.4 Ksec vs 9.2 (Hydra) and 9.0 (FaShapley) for 4-layer CNN, roughly 2-3x faster.
- On Google Pixel 6, 90% compressed 7-layer CNN uses 13.7 MB vs 68.8 MB dense (about 5x less memory) and 3.5 ms vs 6.4 ms inference.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train a compressed network from scratch with VeriCompress instead of FaShapley | 4-layer CNN and 7-layer CNN · CIFAR-10 and SVHN at the reported parameter budgets | standard accuracy: +5.4, +18.5, +3.6, +35.6, +8.3, +13.0, +8.6, +27.9 | Helped |
| Train a compressed network from scratch with VeriCompress instead of FaShapley | 4-layer CNN and 7-layer CNN · CIFAR-10 and SVHN at the reported parameter budgets | verified accuracy: +5.3, +14.2, +1.0, +29.6, +1.6, +5.2, +5.8, +15.7 | Helped |
| Use the 40% or 30% compressed model instead of the dense model | VeriCompress 7-layer CNN · Pedestrian Detection | standard accuracy: 71.6 -> 71.1; 71.6 -> 68.7 | Hurt |
| Use the 40% or 30% compressed model instead of the dense model | VeriCompress 7-layer CNN · Pedestrian Detection | verified accuracy: 36.5 -> 33.6; 36.5 -> 24.2 | Hurt |
| Use the 50% or 40% compressed model instead of the dense model | VeriCompress 7-layer CNN · MNIST | standard accuracy: 97.7 -> 95.4; 97.7 -> 95.1 | Hurt |
| Use the 50% or 40% compressed model instead of the dense model | VeriCompress 7-layer CNN · MNIST | verified accuracy: 87.8 -> 82.2; 87.8 -> 80.1 | Hurt |
| Use VeriCompress instead of Hydra or FaShapley | 4-layer CNN and 7-layer CNN · CIFAR-10 training | training time (Ksec): 9.2, 9.0, 3.4; 20.9, 18.8, 7.5 | Helped |
| Compress the model to 90% instead of using the dense model | VeriCompress 7-layer CNN · Google Pixel 6 deployment | inference time: 6.4 -> 3.5 ms | Helped |
| Compress the model to 90% instead of using the dense model | VeriCompress 7-layer CNN · Google Pixel 6 deployment | memory: 68.8 -> 13.7 MB | Helped |
| Compress the model to 90% instead of using the dense model | VeriCompress ResNet · Google Pixel 6 deployment | inference time: 4.54 -> 0.81 ms | Helped |
| Compress the model to 90% instead of using the dense model | VeriCompress ResNet · Google Pixel 6 deployment | memory: 16.9 -> 2.07 MB | Helped |
Surprising findings
- CROWN-IBP's implicit regularization drives many weights to near zero, so the training loss itself acts as a sparsifying regularizer and no extra regularization term is needed.
- Hydra and FaShapley at high compression on some settings (e.g. 7-layer CNN CIFAR-10 at 10%) collapse to near-chance standard accuracy, while VeriCompress retains 55.1%.
Limitations
- Could not reproduce FaShapley results with provided code, so comparisons use baseline numbers from the FaShapley paper.
- Compression reduces accuracy and verified robustness, a trade-off acknowledged in the cost-benefit analysis.
- Experiments limited to CNN/ResNet/DenseNet/ResNeXt image classifiers and pedestrian detection; no non-image tasks.
Experimental setup
Datasets: CIFAR-10 SVHN MNIST Pedestrian Detection
Benchmarks: CIFAR-10 SVHN MNIST Pedestrian Detection
Metrics: standard accuracy verified accuracy training time memory inference time
Baselines: Hydra FaShapley
Builds on: CROWN-IBP Hydra FaShapley dynamic sparse training
Closest papers by meaning
- A Winning Hand: Compressing Deep Networks Can Improve Out-of-Distribution Robustness (NeurIPS 2021 · similarity 0.84)
- Stress-Testing Neural Network Verifiers with Provably Robust Instances (NeurIPS 2026 · similarity 0.83)
- Forget the Data and Fine-Tuning! Just Fold the Network to Compress (ICLR 2025 · similarity 0.82)
- MCNC: Manifold-Constrained Reparameterization for Neural Compression (ICLR 2025 · similarity 0.82)
- Effective Interplay between Sparsity and Quantization: From Theory to Practice (ICLR 2025 · similarity 0.82)
- Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.