Research map / Point clouds / Point cloud classification
Cloud4D: Estimating Cloud Properties at a High Spatial and Temporal Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Cloud4D reconstructs 4D liquid water content and horizontal winds from six synchronized ground-based cameras using a homography-guided 2D-to-3D transformer, trained on LES-rendered synthetic data.
Problem
Existing satellite and radar instruments cannot observe individual shallow cumulus clouds at high spatial and temporal resolution, limiting validation and improvement of weather and climate models.
Approach
Map ground-camera DINOv2 features to cloud-layer planes via a cloud-specific homography, predict 2.5D cloud properties with a 2D CNN, then refine a full 3D LWC volume with a sparse transformer; wind is obtained by tracking height slices of reconstructed LWC with CoTracker3.
Key results
- Cloud4D achieves LWC MAE 0.03 g/m^3 and LWP MAE 0.06 kg/m^2 against radar, versus 0.13 and 0.39 for VIP-CT, with less than 10% relative error (8.9%).
- Cloud4D provides 25 m spatial and 5 s temporal resolution, versus Sentinel-2 about every five days and MODIS once per day.
- Cloud occupancy F1 of 0.70 vs 0.40 for VIP-CT; CBH MAE 189.58 m and CTH MAE 295.77 m.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add sparse transformer (ablation) | Cloud4D · Real-world radar evaluation | LWC MAE (gm^{-3}): 0.030 -> 0.029 | Helped |
| add sparse transformer (ablation) | Cloud4D · Real-world radar evaluation | CBH MAE (m): 202.04 -> 189.58 | Helped |
| add sparse transformer (ablation) | Cloud4D · Real-world radar evaluation | CTH MAE (m): 299.04 -> 295.77 | Helped |
| add sparse transformer (ablation) | Cloud4D · Real-world radar evaluation | LWP MAE (kgm^{-2}): 0.063 -> 0.063 | No clear effect |
| add sparse transformer (ablation) | Cloud4D · Real-world radar evaluation | Occ F1: 0.70 -> 0.70 | No clear effect |
| drop 2 input views (ablation) | Cloud4D · Real-world radar evaluation | CBH MAE (m): 189.58 -> 316.28 | Hurt |
| drop 2 input views (ablation) | Cloud4D · Real-world radar evaluation | CTH MAE (m): 295.77 -> 371.71 | Hurt |
| drop 2 input views (ablation) | Cloud4D · Real-world radar evaluation | LWC MAE (gm^{-3}): 0.029 -> 0.038 | Hurt |
| drop 2 input views (ablation) | Cloud4D · Real-world radar evaluation | Occ F1: 0.70 -> 0.76 | Helped |
| drop 4 input views (ablation) | Cloud4D · Real-world radar evaluation | CBH MAE (m): 189.58 -> 325.03 | Hurt |
| drop 4 input views (ablation) | Cloud4D · Real-world radar evaluation | CTH MAE (m): 295.77 -> 383.16 | Hurt |
| drop 4 input views (ablation) | Cloud4D · Real-world radar evaluation | LWP MAE (kgm^{-2}): 0.063 -> 0.072 | Hurt |
Surprising findings
- VIP-CT, a satellite-based method, failed to learn cloud property estimation from ground-based cameras.
Limitations
- Focuses on single-layer shallow cumulus clouds; higher layers are occluded and not handled.
- Trained mainly on cumulus clouds; extension to other cloud types untested.
- Ground cameras are susceptible to occlusion from rain, fog, and snow.
Experimental setup
Datasets: MicroHH LES renders Terragen renders Real-world six-camera dataset with radar and wind profiler
Benchmarks: Radar LWC/LWP/CBH/CTH retrievals (real-world, 12 days) Wind profiler comparison Sentinel-2 and MODIS qualitative comparison
Metrics: F1 (cloud occupancy) MAE (LWC, LWP, CBH, CTH) relative error
Baselines: VIP-CT ERA5
Builds on: DINOv2 LoftUp EDM TRELLIS CoTracker3
Closest papers by meaning
- Scaling Storm-Resolving Atmospheric AI Simulation to the Entire Planet (NeurIPS 2026 · similarity 0.85)
- Satellite Observations Guided Diffusion Model for Accurate Meteorological States at Arbitrary Resolution (CVPR 2025 · similarity 0.84)
- Precipitation Downscaling with Spatiotemporal Video Diffusion (NeurIPS 2024 · similarity 0.83)
- PrecipDiff: Leveraging Image Diffusion Models to Enhance Satellite-Based Precipitation Observations (AAAI 2025 · similarity 0.82)
- DeRainGS: Gaussian Splatting for Enhanced Scene Reconstruction in Rainy Environments (AAAI 2025 · similarity 0.82)
- LiDAR4D: Dynamic Neural Fields for Novel Space-time View LiDAR Synthesis (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.