Research map / Point clouds / Segmentation and detection
Equirectangular Point Reconstruction for Domain Adaptive Multimodal 3D Object Detection in Adverse Weather Conditions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EquiDetect, a multimodal LiDAR-camera 3D detector that reconstructs sparse point clouds in equirectangular range images under adverse weather, combined with feature-perturbation domain adaptation and multimodal weather-noise matching for synthetic data.
Problem
Multimodal LiDAR-camera 3D object detection degrades in adverse weather (snow, rain, fog) because weather noise, occlusion and laser attenuation make point clouds sparse, especially at long range; synthetic adverse-weather data are also misaligned between point clouds and images.
Approach
Operates on equirectangular range images with a distance-constrained median denoising that removes weather noise and an object-centric U-Net inpainting ray generator guided by 2D detections to densify distant objects; adds MMD fine-tuning plus iterative adversarial-style feature perturbation (deep SVDD discriminator) for domain adaptation, and radial manipulation of simulated noise points to align
Key results
- On S-KITTI moderate car class, EquiDetect reaches 77.86% (snow), 78.22% (rain), 82.85% (fog) 3DR40 AP, above VirConv-S (72.17, 72.20, 80.23).
- With DA on S-KITTI to Dense, EquiDetect gets 48.84 clear AP vs 44.08 for the best baseline combination (Graph-VoI + NSA), and 52.09 on S-KITTI to CADC snow vs 47.38.
- Multimodal weather noise matching achieves 93.25% (snow) and 94.51% (rain) matching accuracy versus 28.77% and 31.49% for existing simulators.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EquiDetect instead of VirConv-S | 3D object detection · S-KITTI adverse weather | 3DR40 AP (%): 72.17 -> 77.86; 72.20 -> 78.22; 80.23 -> 82.85 | Helped |
| Use EquiDetect without DA instead of VirConv-S | 3D object detection on real-world adverse weather · Dense and CADC | 3DR40 AP (%): 36.32 -> 37.91; 35.32 -> 37.07; 34.92 -> 35.29; 43.18 -> 45.66 | Helped |
| Add EquiDetect's feature perturbations for domain adaptation | EquiDetect · S-KITTI → Dense and CADC | 3DR40 AP (%): 40.09 -> 48.84; 37.91 -> 45.87; 37.07 -> 45.68; 35.29 -> 42.82; 45.66 -> 52.09 | Helped |
| Add the ray generator (ablation) | EquiDetect · S-KITTI (all weather) | 3DR40 AP (%): 80.29 -> 81.71 | Helped |
| Add distance-constrained denoising (ablation) | EquiDetect · S-KITTI (all weather) | 3DR40 AP (%): 76.89 -> 81.71 | Helped |
| Add the auxiliary image restoration network (ablation) | EquiDetect · S-KITTI (all weather) | 3DR40 AP (%): 78.64 -> 81.71 | Helped |
| Add EquiDetect modules beyond the backbone (ablation) | EquiDetect · S-KITTI (all weather) | 3DR40 AP (%): 73.12 -> 81.71 | Helped |
| Match synthetic LiDAR weather noise to image weather particles using radial manipulation | multimodal synthetic weather data · S-KITTI Snow | matching accuracy: 28.77% -> 93.25% | Helped |
| Match synthetic LiDAR weather noise to image weather particles using radial manipulation | multimodal synthetic weather data · S-KITTI Rain | matching accuracy: 31.49% -> 94.51% | Helped |
Limitations
- Parameter count (36.2M) is higher than most multimodal baselines
- Multimodal noise matching evaluated only for snow and rain conditions
- Weather data are simulated for training; real adverse-weather data used only for evaluation and DA
Experimental setup
Datasets: KITTI S-KITTI Dense CADC
Benchmarks: S-KITTI Dense CADC
Metrics: 3DR40 AP FLOPs Params inference time matching accuracy
Baselines: PV-RCNN VoxelNeXt GD-MAE HINTED Focals Conv Graph-VoI SFD TED-M VirConv-S
Builds on: VirConv-S Restormer Graph-VoI MMD domain adaptation Deep SVDD
Closest papers by meaning
- L4DR: LiDAR-4DRadar Fusion for Weather-Robust 3D Object Detection (AAAI 2025 · similarity 0.88)
- Sunshine to Rainstorm: Cross-Weather Knowledge Distillation for Robust 3D Object Detection (AAAI 2024 · similarity 0.87)
- An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains (CVPR 2024 · similarity 0.87)
- UAWTrack: Universal 3D Single Object Tracking in Adverse Weather (AAAI 2025 · similarity 0.86)
- Towards Robust 3D Object Detection with LiDAR and 4D Radar Fusion in Various Weather Conditions (CVPR 2024 · similarity 0.86)
- AS-Det: Active Sampling for Adaptive 3D Object Detection in Point Clouds (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.