Research map / 3D vision / Depth and stereo

MITO: A Millimeter-Wave Dataset and Simulator for Non-Line-of-Sight Perception

NeurIPS 2026 · 3D vision › Depth and stereo · approach: Non-line-of-sight imaging · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MITO, a mmWave SAR dataset of 76 YCB objects (550 LOS/NLOS images at 24 and 77 GHz) plus an open-source mesh-based mmWave simulator, with NLOS segmentation (SAM) and sim-to-real shape classification baselines.

Problem

Optical perception fails under occlusion, and existing mmWave data and simulators cover humans, vehicles, or metallic items rather than diverse everyday objects at object-level resolution, leaving NLOS perception of everyday objects unexplored.

Approach

A robot-mounted dual-frequency (24 and 77 GHz) mmWave radar system fuses frames over a synthetic aperture to form high-resolution 3D images of YCB objects in LOS and NLOS, paired with a simulator that combines specular and edge reflection models with a tunable weighting to mimic different materials.

Key results

• Multi-spectral mmWave segmentation in NLOS achieves a F-score change of only 3.6% from LOS, while the camera achieves 7.0% F-score in NLOS.
• Simulator reaches median 3D F-score of 94% versus real data, vs 90% (specular only) and 71% (edge only).
• Sim2real shape classifier trained only on synthetic data reaches 85.2% overall accuracy (88.8% LOS, 81.2% NLOS).

Surprising findings

• Camera-based SAM achieves only 3.6% precision and 7.0% F-score in NLOS, while mmWave methods exceed it.
• mmWave segmentation recall (63.1%) is much lower than precision (95.1%) due to specular reflections hiding parts of objects.

Limitations

• Specularity causes parts of objects to be missing from mmWave images, lowering recall.
• Dataset covers 76 YCB objects and limited occluders (cardboard, fabric, styrofoam).
• Simulator material modeling is approximated by weighted mixing of two reflection types rather than physical material properties.

Experimental setup

Datasets: YCB

Benchmarks: MITO

Metrics: precision recall F-score IoU accuracy 3D F-score classification accuracy

Baselines: RGB-D camera with SAM specular-only simulation classifier edge-only simulation classifier

Builds on: Segment Anything (SAM) YCB dataset synthetic aperture radar imaging iterative closest point

Closest papers by meaning

• RISE: Single Static Radar-based Indoor Scene Understanding (CVPR 2026 · similarity 0.85)
• X-band Radar Non-Line-of-Sight Imaging (CVPR 2026 · similarity 0.85)
• RadarMOSEVE: A Spatial-Temporal Transformer Network for Radar-Only Moving Object Segmentation and Ego-Velocity Estimation (AAAI 2024 · similarity 0.83)
• MVDoppler-Pose: Multi-Modal Multi-View mmWave Sensing for Long-Distance Self-Occluded Human Walking Pose Estimation (CVPR 2025 · similarity 0.83)
• 3D Point Splatting for mmWave Radar Novel View Synthesis (NeurIPS 2026 · similarity 0.82)
• MultimodalStudio: A Heterogeneous Sensor Dataset and Framework for Neural Rendering across Multiple Imaging Modalities (CVPR 2025 · similarity 0.82)