Research map / Architectures and efficiency / RNNs and normalization
Low-Latency Space-Time Supersampling for Real-Time Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Space-Time Supersampling (STSS), a unified network that jointly performs frame supersampling and extrapolation for real-time rendering, using G-buffers and motion vectors, with random reshading masking and an efficient reshading module.
Problem
Real-time rendering needs both higher resolution and higher frame rate, but existing pipelines run supersampling and frame extrapolation separately, causing redundancy, aliasing, warping-hole artifacts, and high latency.
Approach
Treats aliasing regions and warping holes as unified "reshading regions" and handles them with a single shared network that uses G-buffer and motion vectors; adds Random Reshading Masking (random rectangles added to the occlusion mask) and an Efficient Reshading Module (local attention over nearby regions with similar BRDF features, masking invalid warped pixels).
Key results
- STSSNet reaches 35.02 PSNR on Lewis supersampling frames vs 34.29 for the best two-stage baseline (ExtraNet+NSRR).
- Runs in 4.4 ms per 1080P frame with TensorRT-style reimplementation and 0.4M parameters, versus about 17 ms for the two-stage pipeline (saving up to 75%).
- Ablation: removing RRM lowers Lewis SF PSNR from 35.02 to 34.18 dB; separate optimization of SF and EF lowers it to 34.39.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the unified STSS framework rather than separately optimizing supersampling and extrapolation (ablation) | STSSNet · Lewis (LE), SF and EF | PSNR: 34.39/34.29 -> 35.02/34.72 | Helped |
| Add G-buffer input (ablation) | STSSNet · Lewis (LE), SF and EF | PSNR: 34.32/33.52 -> 35.02/34.72 | Helped |
| Add Random Reshading Masking (RRM) (ablation) | STSSNet · Lewis (LE), SF and EF | PSNR: 34.18/33.98 -> 35.02/34.72 | Helped |
| Add Efficient Reshading Module (ERM) (ablation) | STSSNet · Lewis (LE), SF and EF | PSNR: 34.91/34.62 -> 35.02/34.72 | Helped |
| Add Efficient Reshading Module (ERM) (ablation) | STSSNet · Lewis, edges and warping holes | PSNR: 23.84 / 26.44 -> 23.95 / 26.52 | Helped |
| Use STSSNet instead of ExtraNet+NSRR | STSSNet · Lewis (LE), SF and EF | PSNR: 34.29/33.87 -> 35.02/34.72 | Helped |
| Use STSSNet instead of ExtraNet+NSRR | STSSNet · Subway (SW), SF and EF | PSNR: 30.18/29.88 -> 31.00/30.35 | Helped |
| Use STSSNet instead of ZoomingSloMo | STSSNet · Lewis (LE), SF and EF | LPIPS: 0.050/0.054 -> 0.018/0.020 | Helped |
| Use STSSNet instead of ExtraNet+NSRR | STSSNet · Arena (AR), SF and EF (cross-scene) | PSNR: 33.14/31.97 -> 33.36/32.40 | Mixed |
| Use STSSNet instead of the two-stage ExtraNet+NSRR pipeline | STSSNet · 1080P-target inference | time (ms): 10.3(1.8) / 20.4(15.4) -> 9.0(4.4) | Helped |
Surprising findings
- Swapping the order of the two-stage pipeline (extrapolation after supersampling) degrades performance further, since HR G-buffers are unavailable in the second stage.
Limitations
- Produces artifacts and incorrect predictions under extreme motion and lighting conditions.
- Evaluated only on four synthetic Unreal Engine scenes.
Experimental setup
Datasets: Unreal Engine rendered dataset (Lewis, SunTemple, Subway, Arena)
Benchmarks: Unreal Engine rendered dataset (Lewis, SunTemple, Subway, Arena)
Metrics: PSNR SSIM LPIPS VMAF runtime (ms) parameters
Baselines: RIFE+EDVR RIFE+NSRR ExtraNet+EDVR ExtraNet+NSRR ZoomingSloMo FXAA TAA DLSS
Builds on: NSRR ExtraNet ZoomingSloMo EDVR RIFE
Closest papers by meaning
- Efficient Video Super-Resolution for Real-time Rendering with Decoupled G-buffer Guidance (CVPR 2025 · similarity 0.88)
- High-Quality Real-Time Rendering Using Subpixel Sampling Reconstruction (AAAI 2024 · similarity 0.88)
- Neural Super-Resolution for Real-time Rendering with Radiance Demodulation (CVPR 2024 · similarity 0.86)
- Achieving Lightweight Super-Resolution for Real-Time Computer Graphics (AAAI 2025 · similarity 0.86)
- ReFrame: Layer Caching for Accelerated Inference in Real-Time Rendering (ICML 2025 · similarity 0.84)
- EvSTVSR: Event Guided Space-Time Video Super-Resolution (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.