Research map / Image generation / Autoregressive and latent generation
StyleCineGAN: Landscape Cinemagraph Generation using a Pre-trained StyleGAN
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates one-shot 1024x1024 landscape cinemagraphs from a still image by warping multi-scale deep features of a frozen pre-trained StyleGAN with a learned motion field and mask.
Problem
Automatic cinemagraph generation from a single landscape image is hard to train from scratch and does not scale to high resolution; latent-space-only StyleGAN methods lose fine detail and content.
Approach
Projects the image into both W+ latent codes and StyleGAN intermediate deep features (D10), predicts a static/dynamic mask from deep features, and applies multi-scale deep feature warping (MSDFW) with forward splatting and looping weights inside a frozen StyleGAN.
Key results
- Achieves best static consistency and motion quality vs AL, EMF, DL, T2C; motion FID 39.21 vs 40.12 for T2C (next best).
- User study with 17 participants: preferred over T2C 77.78% (static consistency) and 73.86% (motion quality) in side-by-side comparisons.
- Deep feature inversion (w+ plus D10) reduces FID from 40.83 to 11.82 on GAN inversion versus w+ alone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use StyleCineGAN instead of DL | Landscape cinemagraph generation · Sky Time-Lapse test videos | LPIPS (static consistency): 0.0071 -> 0.0062 | Helped |
| Use StyleCineGAN instead of DL | Landscape cinemagraph generation · Sky Time-Lapse test videos | MS-SSIM (static consistency): 0.9931 -> 0.9962 | Helped |
| Use StyleCineGAN instead of DL | Landscape cinemagraph generation · Sky Time-Lapse test videos | RMSE (static consistency): 2.1044 -> 1.9430 | Helped |
| Use StyleCineGAN instead of T2C | Landscape cinemagraph generation · Sky Time-Lapse test videos | LPIPS (motion quality): 0.0159 -> 0.0131 | Helped |
| Use StyleCineGAN instead of T2C | Landscape cinemagraph generation · Sky Time-Lapse test videos | MS-SSIM (motion quality): 0.7224 -> 0.7237 | Helped |
| Use StyleCineGAN instead of T2C | Landscape cinemagraph generation · Sky Time-Lapse test videos | RMSE (motion quality): 21.2784 -> 20.8948 | Helped |
| Use StyleCineGAN instead of T2C | Landscape cinemagraph generation · Sky Time-Lapse test videos | FID (motion quality): 40.1186 -> 39.2113 | Helped |
| Add forward warping (ablation) | StyleCineGAN · Sky Time-Lapse test videos | MS-SSIM: 0.6853 -> 0.7165 | Helped |
| Add deep feature warping instead of warping RGB images directly (ablation) | StyleCineGAN · Sky Time-Lapse test videos | LPIPS: 0.0537 -> 0.0511 | Helped |
| Warp multi-scale deep features instead of only D10 (ablation) | StyleCineGAN · Sky Time-Lapse test videos | LPIPS: 0.0629 -> 0.0511 | Helped |
| Add a segmentation mask (ablation) | StyleCineGAN · Sky Time-Lapse test videos | MS-SSIM: 0.6980 -> 0.7165 | Helped |
| Use deep feature D10 along with w+ for GAN inversion (ablation) | StyleCineGAN · Sky Time-Lapse and Eulerian images | FID: 40.8276 -> 11.8231 | Helped |
Limitations
- Automatic motion prediction is ambiguous for many landscape images, requiring user hints.
- Cannot isolate motion of very thin structured objects due to low-resolution feature warping.
- Focuses on skies and fluids; other motion types out of scope.
Experimental setup
Datasets: Sky Time-Lapse LHQ Eulerian (EMF) dataset
Benchmarks: Sky Time-Lapse test videos (224) LHQ user study samples
Metrics: LPIPS MS-SSIM RMSE FID user score (1-5) side-by-side preference
Baselines: Animating Landscape (AL) Eulerian Motion Field (EMF) Deep Landscape (DL) Text2Cinemagraph (T2C) MoCoGAN-HD
Builds on: StyleGAN Eulerian Motion Field DatasetGAN MoCoGAN-HD pSp-style encoder (Yao et al. encoder inversion)
Closest papers by meaning
- Analyzing and Improving the Image Quality of StyleGAN (CVPR 2020 · similarity 0.86)
- WarpGAN: Warping-Guided 3D GAN Inversion with Style-Based Novel View Inpainting (NeurIPS 2025 · similarity 0.86)
- Interactive Image Synthesis With Panoptic Layout Generation (CVPR 2022 · similarity 0.86)
- LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators (ICLR 2019 · similarity 0.85)
- WeditGAN: Few-Shot Image Generation via Latent Space Relocation (AAAI 2024 · similarity 0.84)
- Dual Mapping of 2D StyleGAN for 3D-Aware Image Generation and Manipulation (Student Abstract) (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.