Research map / Adversarial robustness and security / Certified and empirical robustness

SGD Jittering: A Training Strategy for Robust and Accurate Model-Based Architectures

ICML 2025 · Adversarial robustness and security › Certified and empirical robustness · approach: Theory of adversarial risk · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SGD jittering, which injects zero-mean Gaussian noise into each gradient-descent update of loop-unrolled model-based architectures for inverse problems, and claims better generalization and average-case robustness than MSE training, with an SPGD extension.

Problem

Robustness and generalization accuracy of model-based architectures (e.g., loop unrolling) for inverse problems are underexplored, and existing robustness methods such as adversarial training or input jittering trade off accuracy and are slow.

Approach

Add small i.i.d. Gaussian noise to each unrolled gradient update during training (noise removed at inference), so the lower-level objective stays unbiased; theory shows this implicitly regularizes the Hessian/gradient of the learned regularizer along the reconstruction trajectory, improving both generalization and average-case robustness relative to MSE training.

Key results

• On seismic deconvolution (GD LU), SGD jittering reaches 35.10 dB PSNR on ID data vs 34.64 for MSE and 33.50 for AT, and 34.93 dB on OOD data.
• On 4x MRI (GD LU), SGD jittering gets 26.77 dB under adversarial attack vs 25.68 for MSE and 27.17 for AT, and 30.36 dB on OOD tumor data vs 29.92 for MSE.
• Training speed: SGD jittering runs at about 4338 it/s on 2D denoising vs 563 it/s for AT, with near-MSE throughput on seismic and MRI.

Measured effects reported in the paper

Surprising findings

• Input jittering (noise on y) degrades in-distribution accuracy substantially for MBAs, while MSE training outperforms SGD jittering on in-distribution data only marginally in some settings.
• Adversarial training yields the best adversarial PSNR on seismic deconvolution but fails to recover a small OOD horizontal layer, treating it as noise.

Limitations

• Theoretical guarantees rely on assumptions (twice-differentiable network, Lipschitz gradient) and bound comparisons to MSE rather than optimal training; the generalization analysis is for denoising with identity forward model.
• Noise variance must be tuned per task (0.01, 0.1, 0.01 chosen), and results on robustness are reported for a limited set of attacks.
• Experiments use small toy problems and a single-coil MRI setting with DnCNN gradient networks; generality to larger architectures is not shown.

Experimental setup

Datasets: fastMRI single-coil knee seismic deconvolution synthetic reflectivity tumor knee MRI (OOD) 2D point denoising synthetic

Benchmarks: fastMRI seismic deconvolution tumor knee OOD MRI 2D point denoising

Metrics: PSNR SSIM worst-case and average-case robustness training throughput (it/s)

Baselines: MSE training adversarial training input jittering Lipschitz regularization (spectral normalization) DDPM

Builds on: adversarial training input jittering stochastic gradient descent loop unrolling

Closest papers by meaning

• Efficient Noise Calculation in Deep Learning-based MRI Reconstructions (ICML 2025 · similarity 0.85)
• Exact Posterior Score Estimation for Solving Linear Inverse Problems (NeurIPS 2026 · similarity 0.85)
• Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models (ICML 2024 · similarity 0.84)
• Ambient Diffusion Posterior Sampling: Solving Inverse Problems with Diffusion Models Trained on Corrupted Data (ICLR 2025 · similarity 0.84)
• The Perception-Robustness Tradeoff in Deterministic Image Restoration (ICML 2024 · similarity 0.84)
• Provable Posterior Sampling with Denoising Oracles via Tilted Transport (NeurIPS 2024 · similarity 0.84)