Research map / Image generation / Fast diffusion sampling
TRIM: Scalable 3D Gaussian Diffusion Inference with Temporal and Spatial Trimming
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TRIM, a training-light post-training framework that accelerates 3D Gaussian diffusion inference via a learned latent trajectory selector (temporal trimming) and corner-reference instance mask denoising with token merging (spatial trimming).
Problem
3D Gaussian diffusion models are slow because of the massive number of Gaussian primitives in denoising and post-denoising (VAE decoding, rendering), which also limits inference-time scaling over multiple sampled trajectories.
Approach
Train a lightweight CNN+MLP latent selector offline (pairwise preference on latents at mid-denoising, supervised by rendered-image CLIP scores) to prune candidate trajectories early, and mask background latent tokens via corner-reference attention with progressive mask expansion, merging them into a single [BG] token, plus opacity correction of background Gaussians after denoising.
Key results
- On T3Bench Single Object, TRIM reaches CLIP Similarity 31.58% and ImageReward 0.12 vs DiffSplat 30.95% and -0.49.
- On GSO image-to-3D, TRIM improves PSNR to 16.78 from DiffSplat's 16.20, SSIM 0.82 vs 0.79, LPIPS 0.17 vs 0.19.
- Runtime on RTX A6000 drops from 8.64 s (SD-3.5-Medium) to 5.24 s with TRIM; FLOPs drop from 195.68T to 106.31T.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply TRIM instead of DiffSplat | 3D text-to-3D generation · T³Bench Single Object | CLIP Similarity (%): 30.95 -> 31.58 | Helped |
| apply TRIM instead of DiffSplat | 3D text-to-3D generation · T³Bench Single Object with Surroundings | CLIP Similarity (%): 30.20 -> 31.48 | Helped |
| apply TRIM instead of DiffSplat | 3D text-to-3D generation · T³Bench Multiple Objects | CLIP Similarity (%): 29.46 -> 30.11 | Helped |
| apply TRIM instead of DiffSplat | 3D image-to-3D reconstruction · GSO | PSNR: 16.20 -> 16.78 | Helped |
| add one convolutional layer to FC2 selector (ablation) | latent selector · T³Bench-Single selector validation | Pairwise accuracy (%): 53.36 -> 74.18 | Helped |
| add one convolutional layer to FC2 selector (ablation) | latent selector · T³Bench-Single | CLIP Similarity (%): 30.83 -> 31.58 | Helped |
| increase selector complexity from Conv1-FC2 to Conv1-FC3 (ablation) | latent selector · T³Bench-Single selector validation | Pairwise accuracy (%): 74.18 -> 73.90 | No clear effect |
| add prompt conditioning to Conv1-FC2 selector (ablation) | latent selector · T³Bench-Single selector validation | Pairwise accuracy (%): 74.18 -> 73.73 | No clear effect |
| add instance masking (ablation) | Stable Diffusion 3.5 Medium inference · inference computation | Runtime (second): 8.64 -> 5.16 | Helped |
| add trajectory reduction (ablation) | Stable Diffusion 3.5 Medium inference · inference computation | FLOPs (T): 195.68 -> 110.07 | Helped |
| add trajectory reduction (ablation) | Stable Diffusion 3.5 Medium inference · inference computation | Runtime (second): 8.64 -> 8.74 | Hurt |
| use trajectory reduction (ablation) | TRIM generation · T³Bench-Single | ImageReward: -0.45 ± 0.05 -> 0.12 ± 0.04 | Helped |
Surprising findings
- Trajectory reduction reduces output diversity (Chamfer Distance upper bound lowers) while raising average quality, suggesting the selector acts as a quality filter that narrows the distribution.
- Increasing denoising steps of DiffSplat beyond a point causes semantic drift and ImageReward decline, while trajectory scaling keeps improving.
Limitations
- Spatial trimming is limited to denoising transformer blocks because the pipeline relies on repurposed 2D backbones.
- Trajectory reduction slightly reduces output diversity.
- Gains on image-to-3D are smaller than on text-to-3D because strong image conditioning reduces trajectory diversity.
Experimental setup
Datasets: G-Objaverse T3Bench Google Scanned Objects
Benchmarks: T3Bench Google Scanned Objects
Metrics: CLIP Similarity CLIP R-Precision ImageReward PSNR SSIM LPIPS Chamfer Distance FLOPs throughput runtime
Baselines: GVGEN DIRECT-3D LGM DiffSplat InstantMesh
Builds on: DiffSplat Stable Diffusion 3.5 DINO Token Merging
Closest papers by meaning
- Communication-Efficient Diffusion Denoising Parallelization via Reuse-then-Predict Mechanism (NeurIPS 2025 · similarity 0.90)
- Efficient Denoising Diffusion via Probabilistic Masking (ICML 2024 · similarity 0.89)
- Reverse Transition Kernel: A Flexible Framework to Accelerate Diffusion Inference (NeurIPS 2024 · similarity 0.89)
- AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models (NeurIPS 2025 · similarity 0.88)
- Fast constrained sampling in pre-trained diffusion models (NeurIPS 2025 · similarity 0.88)
- EVODiff: Entropy-aware Variance Optimized Diffusion Inference (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.