Research map / Image generation / Text-to-image diffusion
Generating Visual Scenes from Touch
Fengyu Yang, Jiacheng Zhang, Andrew Owens
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add touch signal to reflectance-to-image model (ablation) | Visuo-tactile diffusion model · Touch and Go shading estimation | FID: 40.7 -> 36.9 | Improved |
| add touch signal to reflectance-to-image model (ablation) | Visuo-tactile diffusion model · Touch and Go shading estimation | PSNR: 14.5 -> 15.4 | Improved |
| remove hands from generated images (ablation) | Visuo-tactile diffusion model · Touch and Go touch-to-image | FID: 48.7 -> 81.5 | Hurt |
| remove hands from generated images (ablation) | Visuo-tactile diffusion model · Touch and Go touch-to-image | CVTP and material consistency: CVTP 0.12 -> 0.12; material 0.22 -> 0.24 | No clear effect |
| use latent diffusion with touch conditioning instead of Pix2Pix-based VisGel model | touch-to-image generation · Touch and Go | FID: 128.3 -> 48.7 | Improved |
| use latent diffusion instead of Pix2Pix-based VisGel model | touch-to-image generation · VisGel | PSNR: 17.9 -> 21.5 | Improved |
| use diffusion stylization instead of Yang et al. | tactile-driven image stylization · Touch and Go | FID: 22.5 -> 15.8 | Improved |
| condition on touch embedding from contrastive pretraining instead of unconditional or no pretraining (ablation) | touch-to-image generation · Touch and Go | generation quality | Improved |
| provide 5 GelSight frames instead of a single frame (ablation) | touch conditioning · Touch and Go | generation quality | Improved |
| increase denoising starting point N in stylization (ablation) | tactile-driven image stylization · Touch and Go | content preservation versus style match | Mixed |
Closest papers by meaning
- Tactile-Augmented Radiance Fields (CVPR 2024 · similarity 0.90)
- Tactile DreamFusion: Exploiting Tactile Sensing for 3D Generation (NeurIPS 2024 · similarity 0.89)
- Touch2Shape: Touch-Conditioned 3D Diffusion for Shape Exploration and Reconstruction (CVPR 2025 · similarity 0.87)
- TextToucher: Fine-Grained Text-to-Touch Generation (AAAI 2025 · similarity 0.87)
- PhysTacGen: Physics-Aware Visual-Tactile Sensor Image Generation (NeurIPS 2026 · similarity 0.86)
- AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
