Research map / 3D vision / Depth and stereo
RCDN: Towards Robust Camera-Insensitivity Collaborative Perception via Dynamic Feature-based 3D Neural Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RCDN, a dynamic feature-based 3D neural rendering module that recovers failed/noisy camera views in collaborative perception, plus the OPV2V-N dataset, and shows it boosts several baselines under camera failures.
Problem
Collaborative perception assumes all agents' cameras work continuously; in practice cameras can be noisy, obscured, or failed, degrading multi-agent BEV map segmentation and detection.
Approach
Model a collaborative neural rendering field over the shared geometry BEV volume: a hash-grid static background field plus a time-varying dynamic foreground field with scene flow (supervised by optical flow and cycle consistency) that re-renders failed camera views before standard collaborative fusion.
Key results
- With one random camera failure on OPV2V-N, RCDN improves CoBEVT's drivable-area IoU from 32.08 to 47.19 and dynamic vehicle IoU from 32.41 to 55.10 (failure setting).
- Across baselines, RCDN raises average drivable area and lane performance by about 52% and 100% respectively under single-camera failure, per the paper's reported averages.
- Training the dynamic field with the proposed grid/geometry-based representation is reported as about 24x faster than MLP-based dynamic field modeling with higher PSNR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RCDN to collaborative perception baselines | F-Cooper, AttFuse, DiscoNet, V2VNet, and CoBEVT · OPV2V-N camera-track with one random failed camera; Drivable Area | IoU: 28.87/27.99/24.31/27.99/32.08 -> 44.89/44.38/38.54/39.72/47.19 | Helped |
| add RCDN to collaborative perception baselines | F-Cooper, AttFuse, DiscoNet, V2VNet, and CoBEVT · OPV2V-N camera-track with one random failed camera; Lane | IoU: 15.95/18.77/12.29/10.52/14.45 -> 32.23/31.50/22.97/25.24/29.55 | Helped |
| add RCDN to collaborative perception baselines | F-Cooper, AttFuse, DiscoNet, V2VNet, and CoBEVT · OPV2V-N camera-track with one random failed camera; Vehicle | IoU: 29.70/24.76/9.25/11.28/32.41 -> 61.76/52.15/43.03/42.76/55.10 | Helped |
| increase overlapping camera views from +1 to +2 | CoBEVT + RCDN · OPV2V-N, T Section, one failed camera; Drivable Area | IoU: 26.97 -> 26.91 | No clear effect |
| increase overlapping camera views from +1 to +3 | CoBEVT + RCDN · OPV2V-N, T Section, one failed camera; Drivable Area | IoU: 26.97 -> 27.23 | No clear effect |
| increase overlapping camera views from +1 to +3 | CoBEVT + RCDN · OPV2V-N, Midblock, one failed camera; Drivable Area | IoU: 38.87 -> 39.51 | Helped |
| increase overlapping camera views from +1 to +3 | CoBEVT + RCDN · OPV2V-N, T Section, one failed camera; Dynamic Vehicles | IoU: 40.72 -> 42.29 | Helped |
| increase overlapping camera views from +1 to +3 | CoBEVT + RCDN · OPV2V-N, Midblock, one failed camera; Dynamic Vehicles | IoU: 45.60 -> 49.88 | Helped |
| use RCDN-reconstructed views instead of all noisy views | CoBEVT · OPV2V-N; Drivable Area | IoU: 22.65 -> 36.78 | Helped |
| use RCDN-reconstructed views instead of all noisy views | CoBEVT · OPV2V-N; Lanes | IoU: 9.36 -> 20.90 | Helped |
| use RCDN-reconstructed views instead of all noisy views | CoBEVT · OPV2V-N; Dynamic Vehicles | IoU: 12.28 -> 44.54 | Helped |
| use all RCDN-reconstructed views instead of all normal views | V2VNet · OPV2V-N; Drivable Area | IoU: 41.70 -> 32.97 | Hurt |
Surprising findings
- Using all RCDN-reconstructed camera views performs worse than using all normal cameras (e.g., V2VNet drops about 21% on drivable area), attributed to a domain gap between reconstructed and real views.
Limitations
- Relies on manually annotated camera-failure overlap regions and labels, limiting scale.
- Reconstructed views exhibit a domain gap, so replacing all cameras with reconstructed ones degrades performance.
- Evaluated only in simulation (OPV2V-based) on map segmentation with limited scene count.
Experimental setup
Datasets: OPV2V OPV2V-N
Benchmarks: OPV2V-N
Metrics: IoU PSNR AP@0.50 AP@0.70
Baselines: F-Cooper AttFuse DiscoNet V2VNet CoBEVT
Builds on: Instant-NGP NeRF OPV2V CVT CoBEVT
Closest papers by meaning
- RCP-Bench: Benchmarking Robustness for Collaborative Perception Under Diverse Corruptions (CVPR 2025 · similarity 0.87)
- Multi-agent Collaborative Perception via Motion-aware Robust Communication Network (CVPR 2024 · similarity 0.85)
- DSRC: Learning Density-Insensitive and Semantic-Aware Collaborative Representation Against Corruptions (AAAI 2025 · similarity 0.85)
- An Extensible Framework for Open Heterogeneous Collaborative Perception (ICLR 2024 · similarity 0.85)
- What Makes Good Collaborative Views? Contrastive Mutual Information Maximization for Multi-Agent Perception (AAAI 2024 · similarity 0.84)
- FRUC: Feedforward Dynamic Scene Reconstruction from Uncalibrated Collaborative Driving Views (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.