Research map / Video / Video segmentation and tracking
Neural Super-Resolution for Real-time Rendering with Radiance Demodulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces radiance demodulation into real-time rendering super-resolution: super-resolves only the smooth lighting component, re-modulates with a high-resolution material component, and uses an occlusion-aware motion mask plus a ConvLSTM frame-recurrent network with temporal loss.
Problem
Real-time super-resolution rendering struggles to preserve sharp texture details, maintain temporal stability, and avoid ghosting artifacts at high upsampling ratios (4x4) under tight latency budgets.
Approach
Applies radiance demodulation (lighting x material, extended to non-diffuse materials following Zhuang et al.) so the network super-resolves only the smoother lighting component while the HR material component is rendered directly; adds a motion mask from subtracting dual and traditional motion vectors to suppress ghosting, and a ConvLSTM frame-recurrent network with temporal loss.
Key results
- Average PSNR on Bistro 26.43 dB vs 25.42 dB for the best prior baseline (RVRT), with 1.61M parameters and 12.41 ms inference.
- In the ZeroDay scene, the full pipeline takes 29.3 ms versus 89.6 ms for rendering HR radiance directly, a speedup over 3x.
- Radiance demodulation improves PSNR by 1.65 dB over the non-demodulated ablation on the Bistro scene.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use radiance demodulation instead of direct image super-resolution (ablation) | NSRR · Bistro | PSNR (dB): 24.78 -> 25.45 | Helped |
| use radiance demodulation instead of direct image super-resolution (ablation) | BasicVSR++ Unidir. · Bistro | PSNR (dB): 25.03 -> 26.50 | Helped |
| add the frame-recurrent framework (ablation) | Ours · Bistro | PSNR (dB): 26.05 -> 26.20 | Helped |
| add the frame-recurrent framework (ablation) | Ours · Bistro | Warping error: 3.5777 -> 3.9011 | Hurt |
| add temporal loss (ablation) | Ours with frame-recurrent framework · Bistro | Warping error: 3.9011 -> 2.9315 | Helped |
| add temporal loss (ablation) | Ours with frame-recurrent framework · Bistro | PSNR (dB): 26.20 -> 26.43 | Helped |
| add the motion mask (ablation) | Ours · dynamic rendering scenes | ghosting artifacts | Helped |
| use the proposed method instead of the strongest PSNR baseline | real-time rendering super-resolution · Bistro, San M, Square, Bar, ZeroDay, Airplane, and Pica | PSNR (dB) | Mixed |
Surprising findings
- Adding radiance demodulation to BasicVSR++ unidirectional gives 26.50 dB, nearly matching the full method at 26.43 dB, though at about 3x parameters and 6x runtime.
Limitations
- Requires training a separate network per scene for best quality; generalization to unseen scenes trades off quality.
- Relies on G-buffers and a high-resolution material component that must be generated during rendering.
- DLSS 2.0 and FSR 2.0 comparisons limited to 2x2 since they do not officially support 4x4.
Experimental setup
Datasets: Unity-rendered seven-scene dataset (Bistro, Square, San M, Bar, ZeroDay, Airplane, Pica)
Benchmarks: Unity seven-scene rendered test sets Bistro generalization test
Metrics: PSNR SSIM LPIPS VMAF warping error runtime (ms) parameters (M)
Baselines: NSRR BasicVSR++ TTVSR RVRT FRVSR TecoGAN DLSS 2.0 FSR 2.0
Builds on: NSRR Zhuang et al. radiance demodulation for denoising ConvLSTM dual motion vector gated convolution
Closest papers by meaning
- Efficient Video Super-Resolution for Real-time Rendering with Decoupled G-buffer Guidance (CVPR 2025 · similarity 0.86)
- Low-Latency Space-Time Supersampling for Real-Time Rendering (AAAI 2024 · similarity 0.86)
- Achieving Lightweight Super-Resolution for Real-Time Computer Graphics (AAAI 2025 · similarity 0.86)
- PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priors (NeurIPS 2025 · similarity 0.84)
- Ced-NeRF: A Compact and Efficient Method for Dynamic Neural Radiance Fields (AAAI 2024 · similarity 0.84)
- Rad-NeRF: Ray-decoupled Training of Neural Radiance Field (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.