Research map / Adversarial robustness and security / Certified and empirical robustness
Robust Representation Consistency Model via Contrastive Denoising
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes rRCM, which pre-trains a ViT with contrastive instance discrimination along diffusion PF-ODE trajectories and fine-tunes it for one-step denoising-then-classification in randomized smoothing, improving certified accuracy over diffusion-based smoothing at far lower inference cost.
Problem
Diffusion-based randomized smoothing certifies well at small radii but struggles at large l2 perturbations and needs heavy inference compute (e.g., multi-step purification plus majority voting), and uses two separate models for denoising and classification.
Approach
Treat clean and noise-perturbed points along the same PF-ODE sampling trajectory as positive pairs and align their latent representations with an infoNCE-style consistency loss plus a contrastive loss, then fine-tune a single ViT classifier with a consistency and entropy-regularized objective so one forward pass both denoises and classifies.
Key results
- On ImageNet, rRCM-B-Deep reaches 77.4% certified accuracy at r=0 and 25.0% at r=2.5, outperforming DensePure and DiffSmooth at larger radii with 1min 41s latency.
- Versus diffusion-based DDS-style baselines, rRCM improves certified accuracy by 5.3% on average at all radii (up to 11.6% at larger radii), with about 85x lower inference cost on average.
- On CIFAR-10, rRCM-B surpasses DDS by up to 6.4% at r=0.5 and beats diffusion-based methods at all radii.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use rRCM-B-Deep instead of rRCM-B (ablation) | rRCM · ImageNet, certified accuracy across perturbation radii | certified accuracy | Helped |
| Increase training batch size (ablation) | rRCM-B · ImageNet | certified accuracy | Helped |
| Increase model size (ablation) | rRCM · ImageNet | certified accuracy | Helped |
| Restrict pre-training to partial reverse trajectory with T=1 rather than T=80 (ablation) | rRCM-B · CIFAR10, certified accuracy | certified accuracy | Hurt |
| Align noisy points directly with the initial point at t_n=0.5, 1.0, or 2.0 (ablation) | rRCM-B · CIFAR10, certified accuracy | certified accuracy | Hurt |
| Vary the consistency-loss EMA rate (ablation) | rRCM-B · CIFAR10, certified accuracy | certified accuracy | Mixed |
| Vary temperature τ for the consistency and contrastive losses (ablation) | rRCM-B · CIFAR10, certified accuracy | certified accuracy | Mixed |
| Use MoCo-v3 pre-training with Gaussian noise augmentation instead of rRCM trajectory-consistency pre-training (ablation) | ViT-B · CIFAR10, certified accuracy across perturbation radii | certified accuracy | Hurt |
| Increase the majority-voting count (ablation) | DensePure and DiffSmooth · ImageNet, certified accuracy | certified accuracy | Helped |
| Increase majority voting for DensePure (ablation) | DensePure · ImageNet certification latency | latency | Hurt |
| Increase noise level σ for noisy evaluation (ablation) | Linear probe on pre-trained rRCM features · ImageNet and CIFAR10 | linear-probing accuracy | Hurt |
| Use rRCM-B rather than diffusion-based randomized smoothing methods | ImageNet certification · ImageNet, certified accuracy across perturbation radii | certified accuracy: +5.3% on average; up to 11.6% at larger radii | Helped |
Surprising findings
- Training a ViT from scratch with the fine-tuning objective fails to converge, while pre-training with the contrastive denoising objective makes it converge smoothly.
- Models trained to align points directly with the initial clean image (rather than along the trajectory) perform worse as the semantic gap grows.
Limitations
- Certified accuracy is capped at larger noise levels by the semantic similarity between perturbed and clean samples on the stochastic forward trajectory, a limitation shared by diffusion-based methods.
- Evaluation uses only 500 test images per dataset and only ImageNet and CIFAR-10.
- Reported latency gains partly depend on implementation toolkits (xFormers) and batch-size differences versus baselines.
Experimental setup
Datasets: ImageNet CIFAR-10
Benchmarks: ImageNet CIFAR-10
Metrics: certified accuracy at l2 radius inference latency
Baselines: Gaussian Consistency SmoothAdv Boosting MACER SmoothMix DDS DensePure DiffSmooth Denoised
Builds on: randomized smoothing consistency models MoCo-v3 EDM DensePure
Closest papers by meaning
- Truncated Consistency Models (ICLR 2025 · similarity 0.89)
- CAT: Contrastive Adversarial Training for Evaluating the Robustness of Protective Perturbations in Latent Diffusion Models (ICML 2025 · similarity 0.88)
- Adaptive Median Smoothing: Adversarial Defense for Unlearned Text-to-Image Diffusion Models at Inference Time (ICML 2025 · similarity 0.88)
- Certified Robustness for Deep Equilibrium Models via Serialized Random Smoothing (NeurIPS 2024 · similarity 0.88)
- Diffusion Models are Certifiably Robust Classifiers (NeurIPS 2024 · similarity 0.88)
- Improved Diffusion-based Generative Model with Better Adversarial Robustness (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.