Research map / 3D vision / 3D generation
PhysX-3D: Physical-Grounded 3D Asset Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PhysXNet, a 26K-object part-level physics-annotated 3D dataset built with a GPT-4o-plus-human annotation pipeline, and PhysXGen, a dual-branch feed-forward image-to-3D model that generates physical properties alongside TRELLIS geometry.
Problem
Existing 3D generative models focus on geometry and appearance and ignore physical properties (absolute scale, material, affordance, kinematics, function descriptions), and there are no physics-annotated 3D datasets covering these at part level.
Approach
Repurpose a pretrained TRELLIS structured 3D latent space and add a physical VAE and a dual-branch diffusion transformer so physical latents are generated jointly with geometry, with a human-in-the-loop VLM annotation pipeline producing part-level physics labels.
Key results
- PhysXGen improves absolute scale error from 13.21 (TRELLIS+PhysPre) to 7.24 and material from 8.63 to 13.01 on the PhysXNet test set.
- Geometry is preserved or slightly improved versus TRELLIS (PSNR 24.53 vs 24.31; CD 12.7 vs 13.2).
- Against a GPT-4o-based pipeline, PhysXGen reports relative gains of 24% (scale), 64% (material), 28% (kinematics), and 72% (affordance), but is lower on function description (10.11 vs 14.31).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace a separate physical predictor with joint geometry-physics generation | TRELLIS + PhysPre · PhysXNet test set | Absolute scale: 13.21 -> 7.24 | Helped |
| Replace a separate physical predictor with joint geometry-physics generation | TRELLIS + PhysPre · PhysXNet test set | Material: 8.63 -> 13.01 | Helped |
| Replace a separate physical predictor with joint geometry-physics generation | TRELLIS + PhysPre · PhysXNet test set | Description: 6.55 -> 10.11 | Hurt |
| Add geometry-physics dependency in the VAE and diffusion (ablation) | PhysXGen · PhysXNet test set | Absolute scale: 13.21 -> 7.24 | Helped |
| Add geometry-physics dependency in the VAE and diffusion (ablation) | PhysXGen · PhysXNet test set | Material: 8.63 -> 13.01 | Helped |
| Add geometry-physics dependency in the VAE and diffusion (ablation) | PhysXGen · PhysXNet test set | PSNR: 24.31 -> 24.53 | Helped |
| Add geometry-physics dependency in the diffusion model (ablation) | PhysXGen · PhysXNet test set | Absolute scale: 10.57 -> 7.24 | Helped |
| Add geometry-physics dependency in the diffusion model (ablation) | PhysXGen · PhysXNet test set | Material: 9.86 -> 13.01 | Helped |
| Add geometry-physics dependency in the VAE (ablation) | PhysXGen · PhysXNet test set | Absolute scale: 12.01 -> 7.24 | Helped |
| Add geometry-physics dependency in the VAE (ablation) | PhysXGen · PhysXNet test set | PSNR: 24.31 -> 24.53 | Helped |
| Use PhysXGen instead of the GPT-based pipeline | TRE + PartField + GPT · PhysXNet test set | Description: 14.31 -> 10.11 | Hurt |
Surprising findings
- The GPT-4o-based pipeline beats PhysXGen on function description similarity, which the authors attribute to PhysXGen's smaller training set.
Limitations
- Artifacts and limited ability to learn fine-grained properties, with spatial inconsistency in affordance and material maps.
- Poor robustness on extremely large objects in absolute scale prediction.
- Kinematic parent-child prediction uses regression, causing artifacts in hierarchy determination.
Experimental setup
Datasets: PhysXNet PartNet ShapeNet
Benchmarks: PhysXNet test split
Metrics: PSNR Chamfer Distance F-Score Euclidean distance (absolute scale) PSNR on material and affordance maps COV and MMD (kinematics) PSNR on description similarity maps
Baselines: TRELLIS TRELLIS + PhysPre TRELLIS + PartField + GPT-4o
Builds on: TRELLIS CLIP DINOv2 PartNet PartField
Closest papers by meaning
- PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation (NeurIPS 2025 · similarity 0.88)
- PhysGen3D: Crafting a Miniature Interactive World from a Single Image (CVPR 2025 · similarity 0.87)
- OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation (ICLR 2025 · similarity 0.87)
- PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics (CVPR 2024 · similarity 0.87)
- MIDGArD: Modular Interpretable Diffusion over Graphs for Articulated Designs (NeurIPS 2024 · similarity 0.86)
- Structured 3D Latents for Scalable and Versatile 3D Generation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.