Research map / 3D vision / 3D generation
Cue3D: Quantifying the Role of Image Cues in Single-Image 3D Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Cue3D benchmarks seven single-image-to-3D methods on GSO, Toys4K and Zeroverse and quantifies their reliance on image cues (style, shading, texture, silhouette, occlusion, edges, perspective, local continuity) via targeted perturbations.
Problem
It is unclear which monocular image cues modern single-image 3D generators actually exploit, since they are trained end-to-end as black boxes.
Approach
A model-agnostic perturbation framework that applies controlled, cue-specific image edits (style transfer, shading/texture swaps, silhouette dilation and occlusion, edges, orthographic projection, local pixel shuffling, shape CutMix) and measures the resulting drop in 2D and 3D output quality across regression, multi-view and native 3D generators.
Key results
- Native 3D generators (Trellis, Hunyuan3D-2) achieve the best overall CD on GSO (39.64 and 41.82 x1000) and lead on Toys4K, exceeding the next-best method by over 10 points.
- Zeroverse with non-meaningful shapes causes a clear drop in performance across methods; Hunyuan3D-2 overall CD rises from 41.82 (GSO) to 78.09 on Zeroverse.
- Removing shading (gray texture without lighting) raises Hunyuan3D-2 CD by about 30 points on GSO, while keeping shading with gray or checkerboard texture changes CD by under 2 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use meaningful object shapes rather than procedurally assembled meaningless shapes | Single-image 3D generation methods · Zeroverse vs GSO and Toys4K | 3D output quality | Helped |
| Apply shape CutMix to input objects (ablation) | Hunyuan3D-2 · GSO | CD×1000: 20 points | Hurt |
| Apply style transfer that preserves semantics but alters geometric cues (ablation) | Single-image 3D generation methods · GSO and Toys4K | CD×1000 | Hurt |
| Remove shading while retaining the original texture (ablation) | Single-image 3D generation methods · GSO and Toys4K | CD×1000 | Hurt |
| Replace texture while preserving shading (ablation) | Leading methods SF3D, Hunyuan3D-2, and Trellis · GSO and Toys4K | CD×1000 | No clear effect |
| Dilate the object mask (ablation) | Regression-based and multi-view methods · GSO and Toys4K | CD×1000 | Hurt |
| Occlude both the silhouette and image content (ablation) | Single-image 3D generation methods · GSO and Toys4K | CD×1000 | Hurt |
| Use only edge maps (with silhouette) instead of interior image cues (ablation) | Most single-image 3D generation methods · GSO and Toys4K | CD×1000 | Hurt |
| Soften edges with localized Gaussian blur (ablation) | Single-image 3D generation methods · GSO and Toys4K | CD×1000 | Hurt |
| Switch the camera from perspective to orthographic projection (ablation) | Regression-based methods, especially OpenLRM and SF3D · GSO and Toys4K | CD×1000 | Hurt |
| Scramble local image continuity (ablation) | Single-image 3D generation methods · GSO and Toys4K | CD×1000 | Hurt |
| Use directional lighting instead of environment-map lighting (ablation) | Most evaluated methods · GSO | CD×1000 | Hurt |
Surprising findings
- Texture content barely matters for leading models as long as shading is intact, so texture meaningfulness is not necessary for generalization.
- Shape CutMix on a single half-and-half split causes drops over 10 points, larger than most other cues.
Limitations
- Covers only seven methods and two object-centric datasets (GSO, Toys4K).
- Cues are probed mostly in isolation; interplay between multiple cues is left for future work.
- No error bars or statistical tests reported in the main paper.
Experimental setup
Datasets: GSO Toys4K Zeroverse
Benchmarks: GSO Toys4K Zeroverse
Metrics: PSNR SSIM LPIPS Chamfer Distance F-score symmetry F1 DINOv2 similarity
Baselines: LGM OpenLRM CRM SF3D InstantMesh Hunyuan3D-2 Trellis
Builds on: meaningful perturbations CutMix Zeroverse
Closest papers by meaning
- HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D (CVPR 2024 · similarity 0.88)
- Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusion (CVPR 2025 · similarity 0.87)
- MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generation (CVPR 2024 · similarity 0.87)
- Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior (CVPR 2024 · similarity 0.87)
- PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers (NeurIPS 2025 · similarity 0.87)
- Structured 3D Latents for Scalable and Versatile 3D Generation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.