Research map / 3D vision / 3D generation

LiFi: LiDAR Generation from Multi-View Images via Geometric and Semantic Collaborative Guidance

NeurIPS 2026 · 3D vision › 3D generation · approach: Feed-forward 3D asset generation and editing

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

LiFi generates LiDAR range-image scenes from multi-view camera images by guiding a latent diffusion model with a geometry branch (reverse-sampled DepthAnything3 depth/uncertainty/RGB with uncertainty-modulated FiLM encoder) and a cross-view semantic branch, trained with dual-branch balanced classifier-free guidance.

Problem

Generating high-fidelity, geometrically and visually aligned LiDAR scenes from multi-view camera images, a setting where prior methods are not designed for multi-view input and focus on semantic cues or monocular images.

Approach

Two complementary conditioning branches for latent range-image diffusion: a range-view-centered reverse sampling geometry encoder with uncertainty-driven FiLM modulation, and a cross-view semantic encoder with patch, view and extrinsic embeddings, trained with a dual-branch balanced CFG that randomly drops to full, geometry-only, semantic-only, or no conditions.

Key results

• On nuScenes unconditional-vs-conditional LiDAR generation, LiFi reports MMD 2.78e-5 and JSD 3.66e-2, versus 4.57e-5 for LiDM and 6.32e-5 w/o CFG in ablation.
• In camera-to-LiDAR comparison on nuScenes, LiFi achieves CD 0.83 and EMD 50.08 against X-Drive's CD 4.73 and EMD 179.62, with inference time 0.52 s/frame.
• Downstream BEVFusion with generated LiDAR reaches mAP 0.526 versus 0.529 with real LiDAR and 0.278 for camera-only BEVDet.

Surprising findings

• Downstream detection with generated LiDAR (mAP 0.526) nearly matches real LiDAR (0.529), though the authors caution this does not show generated scenes independently support perception.

Limitations

• Range image representation is not lossless, omitting some camera image details in generated LiDAR.
• Architecture is not designed for explicit temporal modeling; consecutive-frame consistency is emergent rather than enforced.
• Downstream detection evaluation does not show generated LiDAR can independently support perception comparable to real data.

Experimental setup

Datasets: nuScenes KITTI-360

Benchmarks: nuScenes KITTI-360

Metrics: MMD JSD FRD FPD CD EMD DAS CTC TTCE mAP

Baselines: LiDARGen RangeLDM LiDM UniScene OpenDWM LiDARCrafter X-Drive

Builds on: latent diffusion model RangeLDM classifier-free guidance FiLM

Closest papers by meaning

• Spiral: Semantic-Aware Progressive LiDAR Scene Generation and Understanding (NeurIPS 2025 · similarity 0.88)
• Towards Realistic Scene Generation with LiDAR Diffusion Models (CVPR 2024 · similarity 0.87)
• LiSu: A Dataset and Method for LiDAR Surface Normal Estimation (CVPR 2025 · similarity 0.87)
• AlignMiF: Geometry-Aligned Multimodal Implicit Field for LiDAR-Camera Joint Synthesis (CVPR 2024 · similarity 0.87)
• PC-BEV: An Efficient Polar-Cartesian BEV Fusion Framework for LiDAR Semantic Segmentation (AAAI 2025 · similarity 0.87)
• GeoNLF: Geometry guided Pose-Free Neural LiDAR Fields (NeurIPS 2024 · similarity 0.86)