Research map / 3D vision / 3D generation
Seeing in the Dark: Benchmarking Egocentric 3D Vision with the Oxford Day-and-Night Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Oxford Day-and-Night, a city-scale egocentric dataset captured with Meta ARIA glasses and aligned to TLS ground truth, and benchmarks visual relocalization and in-the-wild NVS under day/night lighting.
Problem
Existing 3D vision datasets lack the combination of egocentric 6DoF head motion, full-color imagery, ground-truth 3D geometry, and wide day-to-night lighting variation needed to evaluate NVS and visual relocalization for smart glasses in low light.
Approach
Use multi-session SLAM (Meta MPS) on ARIA glasses recordings across day, dusk and night, and align the resulting trajectories and point clouds to terrestrial laser scans from Oxford Spires via AprilTags and ICP, yielding egocentric day-night data with centimeter-level ground truth geometry and two derived benchmarks.
Key results
- RoMA with HLoc and NetVLAD reaches only 25.24% accuracy at (0.25m, 2°) on Bodleian night queries, versus 80.57% with ground-truth pose retrieval.
- SCR methods ACE and GLACE fail almost entirely on night queries (near 0% at the strictest threshold).
- In-the-wild 3DGS methods reach PSNR above 25 on most scenes but LPIPS above 0.5, indicating poor perceptual quality.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use feature matching rather than scene coordinate regression | Visual relocalization methods · Oxford Day-and-Night day and night queries | localization accuracy | Helped |
| Use MegaLoc instead of NetVLAD for RoMA retrieval | RoMA on night queries · Bodleian Library | localization accuracy (0.25m, 2°): 25.24 -> 70.25 | Helped |
| Use MegaLoc instead of NetVLAD for RoMA retrieval | RoMA on night queries · H.B. Allen Centre | localization accuracy (0.25m, 2°): 57.91 -> 66.37 | Helped |
| Use MegaLoc instead of NetVLAD for RoMA retrieval | RoMA on night queries · Keble College | localization accuracy (0.25m, 2°): 14.96 -> 31.50 | Helped |
| Use MegaLoc instead of NetVLAD for RoMA retrieval | RoMA on night queries · Observatory Quarter | localization accuracy (0.25m, 2°): 58.94 -> 72.06 | Helped |
| Use MegaLoc instead of NetVLAD for RoMA retrieval | RoMA on night queries · Robotics Institute | localization accuracy (0.25m, 2°): 72.34 -> 78.22 | Helped |
| Replace Gaussian-Wild with Splatfacto-W | 3DGS in-the-wild NVS · H.B. Allen Centre | PSNR: 24.94 -> 25.65 | Helped |
| Replace Gaussian-Wild with Splatfacto-W | 3DGS in-the-wild NVS · Bodleian Library, Keble College, Observatory Quarter, and Robotics Institute | PSNR | Hurt |
Surprising findings
- PSNR fails to reflect perceptual NVS quality: methods with PSNR above 25 still show LPIPS above 0.5.
- Better image retrieval (MegaLoc, GT pose retrieval) lifts RoMA night accuracy far more than the choice of matcher, suggesting retrieval is the main bottleneck.
Limitations
- Ground-truth poses depend on Aria MPS multi-session SLAM accuracy, which is hard to verify at city scale.
- AprilTag detection and registration add error to the ground-truth alignment.
- Current in-the-wild 3DGS methods struggle at this scale, forcing aggressive temporal and spatial subsampling.
Experimental setup
Datasets: Oxford Spires Oxford Day-and-Night
Benchmarks: Oxford Day-and-Night visual relocalization Oxford Day-and-Night NVS
Metrics: percentage of queries localized within (0.25m,2°)/(0.5m,5°)/ PSNR LPIPS SSIM point-to-point distance
Builds on: Oxford Spires 3DGS HLoc
Closest papers by meaning
- Sparse Views Near Light: A Practical Paradigm for Uncalibrated Point-light Photometric Stereo (CVPR 2024 · similarity 0.85)
- See through the Dark: Learning Illumination-affined Representations for Nighttime Occupancy Prediction (NeurIPS 2025 · similarity 0.85)
- Scalable Benchmarking and Robust Learning for Noise-Free Ego-Motion and 3D Reconstruction from Noisy Video (ICLR 2025 · similarity 0.85)
- 3D Human Pose Perception from Egocentric Stereo Videos (CVPR 2024 · similarity 0.84)
- EgoSim: An Egocentric Multi-view Simulator and Real Dataset for Body-worn Cameras during Motion and Activity (NeurIPS 2024 · similarity 0.84)
- Gaze Beyond the Frame: Forecasting Egocentric 3D Visual Span (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.