Research map / 3D vision / NeRF and radiance fields
You Always Recognize Me (YARM): Robust Texture Synthesis Against Multi-View Corruption
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Optimizes the color voxel grid of a 3D object reconstructed from multi-view images, via adversarial-style perturbation under random corruptions, to create a robust texture (and a universal texture across same-class objects) that improves classifier accuracy on corrupted images without fine-tuning the model.
Problem
Pretrained image classifiers degrade under unknown image corruptions (noise, blur, weather, compression), and existing restoration or fine-tuning fixes either fail on complex corruptions or hurt clean-image accuracy.
Approach
Treat object appearance as a design variable: optimize a perturbation on the fixed-geometry DVGO-style voxel color grid so that rendered views remain recognizable under random corruptions, and share one perturbation across same-class objects to get a universal robust texture; the surrogate choice (smaller models) affects transferability.
Key results
- On the VGG16 proxy, single-object robust texture reaches 0.8540 random-severity accuracy vs 0.1496 on clean-trained clean-data baseline and 0.7406 for VQSA fine-tuning, with relative mCE 0.1878.
- Universal texture reaches 0.3247 average accuracy under random severity on VGG16 proxy, below the single-object texture but above URIE and UnAdv.
- Transferred single-object textures from VGG16 improve average accuracy on ResNet-50, MobileNetV2, Inception-V3 and others (e.g., 0.6019 on ResNet-50), but universal textures gave smaller transfer gains.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| optimize a single object's texture instead of using VQSA fine-tuning | VGG16 · IM3D, random corruption severity | accuracy: 0.7406 -> 0.8540 | Helped |
| optimize a single object's texture instead of using VQSA fine-tuning | ResNet-18 · IM3D, random corruption severity | accuracy: 0.8051 -> 0.8427 | Helped |
| use a universal texture rather than a single-object texture | YARM with VGG16 proxy · IM3D, random corruption severity | accuracy: 0.8540 -> 0.3247 | Hurt |
| transfer single-object robust textures generated with ResNet-18 | YARM · IM3D, severity 1–5, ResNet-50 | average accuracy: 0.3472 -> 0.6684 | Helped |
| transfer single-object robust textures generated with ResNet-18 | YARM · IM3D, severity 1–5, ResNet-152 | average accuracy | Helped |
| increase the perturbation bound from ϵ=1.0 to ϵ=5.0 (ablation) | YARM with VGG16 proxy · IM3D, severity 1–5 | average accuracy: 0.6857 -> 0.8513 | Helped |
| increase the perturbation bound from ϵ=1.0 to ϵ=5.0 (ablation) | YARM with ResNet-18 proxy · IM3D, severity 1–5 | average accuracy: 0.6531 -> 0.8481 | Helped |
| increase voxel grid resolution from 160 to 320 at ϵ=5.0 (ablation) | YARM with VGG16 proxy · IM3D, severity 1–5 | average accuracy: 0.8513 -> 0.8492 | Hurt |
| increase voxel grid resolution from 160 to 320 at ϵ=5.0 (ablation) | YARM with ResNet-18 proxy · IM3D, severity 1–5 | average accuracy: 0.8481 -> 0.8504 | Helped |
| increase voxel grid resolution from 160 to 320 at ϵ=1.0 (ablation) | YARM with VGG16 proxy · IM3D, severity 1–5 | average accuracy: 0.6857 -> 0.7063 | Helped |
| increase voxel grid resolution from 160 to 320 at ϵ=1.0 (ablation) | YARM with ResNet-18 proxy · IM3D, severity 1–5 | average accuracy: 0.6531 -> 0.6762 | Mixed |
| use ResNet-18 rather than VGG16 as the proxy model | YARM single-object texture · IM3D, transfer to other architectures | average accuracy | Mixed |
Surprising findings
- Robust textures optimized with smaller surrogate models (ResNet-18) transfer better than larger surrogates across ResNet variants, reversing the usual adversarial transferability intuition.
- Several baselines (DCP, UnAdv) reduced accuracy relative to the original model, and for high-robustness models like ResNet-152, ViT-B-16 and Swin-Small most methods hurt performance.
Limitations
- Universal texture generalizes less well across architectures than single-object textures.
- Requires multi-view images and 3D voxel reconstruction for each object, and only the color grid is optimized while geometry is fixed.
- Evaluated only on rendered IM3D objects and a fixed set of 15 synthetic corruptions, not real-world captured degradations.
Experimental setup
Datasets: IM3D ImageNet-C style corruptions (15 types, 5 severities)
Benchmarks: IM3D
Metrics: accuracy mean corruption error relative mCE average accuracy
Baselines: URIE VQSA DCP UnAdv
Builds on: DVGO Unadversarial examples (Salman et al., 2021) ImageNet-C corruption protocol universal adversarial perturbations
Closest papers by meaning
- ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object (CVPR 2024 · similarity 0.83)
- Revisiting Visual Corruptions in LVLMs: A Shape-Texture Perspective on Model Failures (CVPR 2026 · similarity 0.83)
- SeD: Semantic-Aware Discriminator for Image Super-Resolution (CVPR 2024 · similarity 0.82)
- LIRM: Large Inverse Rendering Model for Progressive Reconstruction of Shape, Materials and View-dependent Radiance Fields (CVPR 2025 · similarity 0.82)
- LAION-C: An Out-of-Distribution Benchmark for Web-Scale Vision Models (ICML 2025 · similarity 0.82)
- RGB-to-Polarization Estimation: A New Task and Benchmark Study (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.