Research map / Video / Video segmentation and tracking
Leveraging Frame Affinity for sRGB-to-RAW Video De-rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FAGDN, a chained video de-rendering network that reconstructs RAW frames from sRGB video plus a single saved RAW frame, using optical-flow warping and global-local feature aggregation, and introduces the RVD benchmark.
Problem
Recovering RAW video from sRGB video without the per-frame metadata sampling that single-image de-rendering methods require, since per-frame sampling is computationally burdensome on cameras for video.
Approach
Save only one RAW frame at capture time and chain-reconstruct subsequent RAW frames by warping the previous predicted RAW with optical flow, refining it with a confidence map and a Global-and-Local Feature Aggregation module that fuses scene-specific and position-specific cues from the previous frame.
Key results
- On RVD-Part2, the proposed method reaches 49.71 PSNR / 0.9983 SSIM versus 46.25 PSNR for INF (metadata-based) and 42.25 for CAM.
- On RVD-Part1, the method reaches 48.86 PSNR / 0.9980 SSIM, above INF at 47.67 PSNR without any per-image metadata.
- Ablation on RVD-Part2 shows removing temporal affinity prior extraction (warping) drops PSNR by 6.17 dB (42.60 vs 49.71 for the full model).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use frame affinity-guided de-rendering without per-image metadata | sRGB-to-RAW video de-rendering · RVD-Part1 | PSNR: 41.87 -> 48.86 | Helped |
| use frame affinity-guided de-rendering without per-image metadata | sRGB-to-RAW video de-rendering · RVD-Part2 | PSNR: 42.26 -> 49.71 | Helped |
| use FAGDN instead of INF | sRGB-to-RAW video de-rendering · RVD-Part1 low resolution | PSNR: 45.50 -> 47.90 | Helped |
| use FAGDN instead of INF | sRGB-to-RAW video de-rendering · RVD-Part1 fast motion | PSNR: 45.79 -> 46.36 | Helped |
| use FAGDN instead of INF | sRGB-to-RAW video de-rendering · RVD-Part1 dark lighting | PSNR: 51.42 -> 53.60 | Helped |
| add temporal affinity prior extraction using warping (ablation) | FAGDN · RVD-Part2 | PSNR: 42.60 -> 49.71; decreased by 6.17 without this step | Helped |
| add confidence map M (ablation) | FAGDN · RVD-Part2 | PSNR: 47.68 -> 49.71 | Helped |
| add Global and Local Feature Aggregation (GLA) (ablation) | FAGDN · RVD-Part2 | PSNR: 46.12 -> 49.71 | Helped |
| add auxiliary loss on intermediate outputs (ablation) | FAGDN · RVD-Part2 | PSNR: 47.91 -> 49.71 | Helped |
| add auxiliary loss on intermediate outputs (ablation) | FAGDN · RVD-Part2 | SSIM: 0.9983 -> 0.9983 | No clear effect |
Surprising findings
- Without any per-image metadata, the method outperforms metadata-based INF and CAM on both RVD subsets.
- Quality degrades noticeably over long chains (beyond about 100 frames), so error accumulation limits the chained approach.
Limitations
- Quality declines over long frame chains due to accumulated information loss.
- Fast motion remains challenging because the single reference RAW frame provides limited information.
- Models are trained and tested separately per device subset, so cross-camera generalization is not shown.
Experimental setup
Datasets: RVD-Part1 RVD-Part2
Benchmarks: RVD-Part1 RVD-Part2
Metrics: PSNR SSIM
Baselines: CAM CAM w/ fine-tuning INF
Builds on: CAM INF RAFT
Closest papers by meaning
- Learning sRGB-to-Raw-RGB De-Rendering With Content-Aware Metadata (CVPR 2022 · similarity 0.91)
- RGB-Only Supervised Camera Parameter Optimization in Dynamic Scenes (NeurIPS 2025 · similarity 0.83)
- DiffRAW: Leveraging Diffusion Model to Generate DSLR-Comparable Perceptual Quality sRGB from Smartphone RAW Images (AAAI 2024 · similarity 0.83)
- ISPDiffuser: Learning RAW-to-sRGB Mappings with Texture-Aware Diffusion Models and Histogram-Guided Color Consistency (AAAI 2025 · similarity 0.83)
- BARD-GS: Blur-Aware Reconstruction of Dynamic Scenes via Gaussian Splatting (CVPR 2025 · similarity 0.83)
- Blind Bitstream-corrupted Video Recovery via Metadata-guided Diffusion Model (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.