Research map / 3D vision / Human pose estimation
RayTran: 3D Pose Estimation and Shape Reconstruction of Multiple Objects from Videos with Ray-Traced Transformers
Michał J. Tyszkiewicz, Kevis-Kokitsi Maninis, Stefan Popov, Vittorio Ferrari
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add 2D FG/BG segmentation auxiliary task (ablation) | RayTran · Scan2CAD val | class avg. accuracy: -2.4% (36.2% vs. 33.8%) | Improved |
| add 2D FG/BG segmentation auxiliary task (ablation) | RayTran · Scan2CAD val | global avg. accuracy: 40.1 -> 43.0 | Improved |
| give ground-truth 2D masks as a 4th input channel (ablation) | RayTran · Scan2CAD val | class avg. accuracy: +11.4% (reaching 47.6%) | Improved |
| give ground-truth 2D masks as a 4th input channel (ablation) | RayTran · Scan2CAD val | global avg. accuracy: +9.5% (reaching 52.5%) | Improved |
| restrict evaluation to scenes with at most 10 objects (ablation) | RayTran · Scan2CAD val subsets | class avg. accuracy: 36.2% -> 37.4% | Improved |
| restrict evaluation to scenes with at most 10 objects (ablation) | Vid2CAD · Scan2CAD val subsets | class avg. accuracy: 30.7% | No clear effect |
| use 48 instead of 96 inference frames (ablation) | RayTran · Scan2CAD val | class avg. accuracy: -0.4% | Hurt |
| use 20 instead of 96 inference frames (ablation) | RayTran · Scan2CAD val | class avg. accuracy: -3.3% | Hurt |
| single-stage ray-traced transformer vs tracking-based Vid2CAD | RayTran · Scan2CAD val | class avg. accuracy: +5.5% | Improved |
| RayTran vs MVS + RGB-D fitter | RayTran · Scan2CAD val | class avg. accuracy: +17.4% | Improved |
Later papers that compared with this method
1 later accepted paper reports RayTran as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- FastCAD: Real-Time CAD Retrieval and Alignment from Scans and Videos (ECCV 2024 · better on 1 of 1 shared result)
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Vid2CAD — Vid2CAD: CAD Model Alignment Using Multi-View Constraints From Videos (TPAMI 2022) · better on 4 of 4 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- TransformerFusion: Monocular RGB Scene Reconstruction using Transformers (NeurIPS 2021 · similarity 0.89)
- 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction (ECCV 2016 · similarity 0.88)
- CoReNet: Coherent 3D Scene Reconstruction from a Single RGB Image (ECCV 2020 · similarity 0.88)
- VGGT: Visual Geometry Grounded Transformer (CVPR 2025 · similarity 0.86)
- CRAYM: Neural Field Optimization via Camera RAY Matching (NeurIPS 2024 · similarity 0.86)
- Cameras as Rays: Pose Estimation via Ray Diffusion (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
