Research map / Video / Video segmentation and tracking
Efficient Video Super-Resolution for Real-time Rendering with Decoupled G-buffer Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RDG, an asymmetric UNet video super-resolution network that decouples G-buffer guidance (normal/BRDF via HFB, depth and motion vectors via CTR) for real-time rendering, reaching 126 FPS at 1080P with RDG-s.
Problem
Real-time rendering super-resolution needs high-quality, temporally stable upsampling at low latency, but existing VSR methods use bidirectional propagation and flow warping, and existing G-buffer-guided methods concatenate G-buffers indiscriminately or rely on radiance demodulation that causes arti
Approach
Asymmetric UNet with deformable-sampling encoder blocks (DFM), decoder that injects normal/BRDF high-frequency detail via cross-gating (HFB), and a cross-frame refiner that warps history with motion vectors and corrects with depth-aware partial cross attention (CTR), instead of concatenating G-buffers as raw input.
Key results
- RDG achieves 29.69 average PSNR and 88.96 average VMAF, outperforming NSRR by 0.35 dB and NSRD and FuseSR by 3.4 dB and 1.95 dB on average.
- RDG-s (0.304M params) reaches 29.37 dB average PSNR and 87.22 VMAF with 7.93 ms latency, and produces 1080P results at 126 FPS on an RTX 3090.
- Removing all G-buffers drops RDG-s PSNR by 0.47 dB and VMAF by 1.31; removing the motion vector input drops PSNR by 0.37 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DFM instead of using ResBlock in the encoder (ablation) | RDG-s · Rendering test datasets (average) | PSNR: 29.17 -> 29.37 | Helped |
| Use deformable sampling instead of pooling+DW3×3 (ablation) | RDG-s · Rendering test datasets (average) | PSNR: 29.21 -> 29.37 | Helped |
| Remove all G-buffer guidance (ablation) | RDG-s · Rendering test datasets (average) | PSNR: 28.90 -> 29.37 | Helped |
| Add normal guidance (ablation) | RDG-s · Rendering test datasets (average) | PSNR: 29.27 -> 29.37 | Helped |
| Add BRDF guidance (ablation) | RDG-s · Rendering test datasets (average) | PSNR: 29.11 -> 29.37 | Helped |
| Add motion-vector guidance to CTR (ablation) | RDG-s · Rendering test datasets (average) | PSNR: 29.00 -> 29.37 | Helped |
| Add depth guidance to CTR (ablation) | RDG-s · Rendering test datasets (average) | VMAF: 87.01 -> 87.22 | Helped |
| Use decoupled G-buffer guidance instead of concatenating G-buffer components (ablation) | RDG-s · Rendering test datasets (average) | PSNR: 29.29 -> 29.37 | Helped |
| Use HFB instead of ResBlock (ablation) | RDG-s · Rendering test datasets (average) | PSNR: 29.05 -> 29.37 | Helped |
| Use HFB instead of SFT layer (ablation) | RDG-s · Rendering test datasets (average) | PSNR: 29.29 -> 29.37 | Helped |
| Use CTR instead of ResBlock (ablation) | RDG-s · Rendering test datasets (average) | VMAF: 85.27 -> 87.22 | Helped |
| Use CTR instead of ConvGRU (ablation) | RDG-s · Rendering test datasets (average) | VMAF: 86.39 -> 87.22 | Helped |
Surprising findings
- Directly concatenating G-buffers with LR frames (as in NSRR-style UNets) underperforms the decoupled design, with PSNR falling to 28.80 dB in the UNet variant versus 29.37 dB for RDG-s.
- Radiance-demodulation methods (NSRD, FuseSR) show re-modulation artifacts on translucent and anisotropic materials.
Limitations
- Baselines were retrained on the authors' own rendering dataset, so results depend on that data rather than public benchmarks.
- G-buffer inputs (normal, BRDF, depth, motion vector) must be available from the renderer, which limits applicability to pipelines that do not expose them.
Experimental setup
Datasets: in-house Cycles rendering dataset (180 clips) Bar Bistro NYC Forest Square Square-night Tenshu ZeroDay
Benchmarks: Bar Bistro NYC Forest Square Square-night Tenshu ZeroDay
Metrics: PSNR SSIM VMAF latency (ms) GPU memory parameters
Baselines: SAFMN SMFANet BasicVSR BasicVSR++ NSRR NSRD FuseSR
Builds on: BasicVSR++ SAFMN NSRR Charbonnier loss
Closest papers by meaning
- Achieving Lightweight Super-Resolution for Real-Time Computer Graphics (AAAI 2025 · similarity 0.89)
- Low-Latency Space-Time Supersampling for Real-Time Rendering (AAAI 2024 · similarity 0.88)
- Neural Super-Resolution for Real-time Rendering with Radiance Demodulation (CVPR 2024 · similarity 0.86)
- Temporal Smoothness-Aware Rate-Distortion Optimized 4D Gaussian Splatting (NeurIPS 2025 · similarity 0.85)
- 4K4D: Real-Time 4D View Synthesis at 4K Resolution (CVPR 2024 · similarity 0.85)
- 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.