Research map / Image generation / Text-to-image diffusion
Generative Photography: Scene-Consistent Camera Control for Realistic Text-to-Image Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Generative Photography, which controls camera intrinsics (bokeh, focal length, shutter speed, color temperature) in text-to-image generation by lifting T2I to a T2V model and training on a physically simulated differential dataset with a differential camera encoder.
Problem
Text-to-image diffusion models cannot interpret camera intrinsic settings (focal length, aperture/bokeh, shutter speed, color temperature) and fail to keep the scene consistent when those settings change.
Approach
Dimensionality lifting treats camera-setting variation as a frame dimension of a text-to-video model so the scene prompt is shared across frames, and a differential camera encoder feeds both camera-setting embeddings and differences between adjacent settings, trained on a physically simulated differential dataset built from captioned base images with randomly sampled continuous settings.
Key results
- On bokeh rendering, the method reaches CorrCoef 0.8626 versus 0.2492 for SD3 and 0.6025 for fine-tuned CameraCtrl.
- On focal length, the method reaches CorrCoef 0.9695 versus 0.8671 for fine-tuned CameraCtrl and 0.2356 for SD3.
- Prompt-following CLIP scores stay comparable across methods (about 0.29 to 0.30), while SD3 and FLUX show lower camera accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed camera-control framework | AnimateDiff-based generative photography · Bokeh rendering | CorrCoef: 0.6025 -> 0.8626 | Helped |
| use the proposed camera-control framework | AnimateDiff-based generative photography · Focal length | CorrCoef: 0.8671 -> 0.9695 | Helped |
| use the proposed camera-control framework | AnimateDiff-based generative photography · Shutter speed | CorrCoef: 0.7526 -> 0.9264 | Helped |
| use the proposed camera-control framework | AnimateDiff-based generative photography · Color temperature | CorrCoef: 0.5812 -> 0.8970 | Helped |
| add inter-frame difference features (ablation) | Generative Photography · Bokeh rendering | CorrCoef: 0.7631 -> 0.8626 | Helped |
| add inter-frame difference features (ablation) | Generative Photography · Focal length | CorrCoef: 0.8594 -> 0.9695 | Helped |
| add inter-frame difference features (ablation) | Generative Photography · Shutter speed | CorrCoef: 0.8864 -> 0.9264 | Helped |
| add inter-frame difference features (ablation) | Generative Photography · Color temperature | CorrCoef: 0.8586 -> 0.8970 | Helped |
| use continuous rather than discrete uniform sampling (ablation) | Generative Photography · Bokeh rendering | CorrCoef: 0.6774 -> 0.8626 | Helped |
| use continuous rather than discrete uniform sampling (ablation) | Generative Photography · Focal length | CorrCoef: 0.7254 -> 0.9695 | Helped |
| use continuous rather than discrete uniform sampling (ablation) | Generative Photography · Shutter speed | CorrCoef: 0.8384 -> 0.9264 | Helped |
| use continuous rather than discrete uniform sampling (ablation) | Generative Photography · Color temperature | CorrCoef: 0.8140 -> 0.8970 | Helped |
Surprising findings
- Continuous random sampling of camera values outperforms uniform discrete sampling in accuracy and consistency (e.g., bokeh CorrCoef 0.8626 vs 0.6774).
Limitations
- Evaluation relies on the authors' physically simulated reference rather than real camera captures.
- The LPIPS consistency metric is ambiguous because intended camera changes also alter frame differences.
- Only 5 frames per generation are used to limit compute, and more frames increase cost.
Experimental setup
Datasets: own photography public image datasets (Unsplash-style base images, refs [42, 72]) all-in-focus images from [14, 20, 73]
Benchmarks: internal differential test set (75 camera-setting sets per task)
Metrics: Pearson correlation coefficient (accuracy to physical trend) frame-wise LPIPS (consistency) CLIP score (prompt following)
Baselines: Stable Diffusion 3 FLUX AnimateDiff (w/ and w/o fine-tuning) CameraCtrl (w/ and w/o fine-tuning)
Builds on: AnimateDiff T2I-Adapter CameraCtrl CLIP Depth Anything
Closest papers by meaning
- PreciseCam: Precise Camera Control for Text-to-Image Generation (CVPR 2025 · similarity 0.90)
- 3D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation (CVPR 2024 · similarity 0.88)
- 4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models (NeurIPS 2024 · similarity 0.88)
- Generative Photomontage (CVPR 2025 · similarity 0.87)
- Learning Visual Generative Priors without Text (CVPR 2025 · similarity 0.87)
- ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.