Research map / Video / Video segmentation and tracking
PNeRV: Enhancing Spatial Consistency via Pyramidal Neural Representation for Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PNeRV, a pyramidal implicit neural video representation that uses a low-cost Kronecker fully-connected upscaling layer (KFC) and a gated Benign Selective Memory (BSM) fusion to add multi-scale shortcuts, improving spatial consistency and perceptual quality.
Problem
Current NeRV-style implicit video representations suffer from spatial inconsistency and poor perceptual quality, partly due to lacking global receptive field and multi-scale information flow in their decoders.
Approach
Replace subpixel/conv upsampling with a Kronecker fully-connected layer that gives global, parameter-cheap upscaling for shortcuts from high-level embeddings, and fuse them into each decoder layer via a gated BSM unit, forming a pyramidal decoder; also gives a Universal Approximation Theory analysis of NeRV systems.
Key results
- On UVG, PNeRV-L averages 36.12 dB PSNR versus 31.63 dB for NeRV (+4.49 dB per the abstract), with a 231% FVD gap over NeRV.
- On DAVIS, PNeRV-L reaches 31.27 dB PSNR / 0.908 SSIM average, +0.88 dB over DiffNeRV and +3.28 dB over NeRV.
- PNeRV reports LPIPS 0.163 on UVG (vs 0.176 for DiffNeRV) and 0.285 on DAVIS (vs 0.326 for DiffNeRV).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PNeRV-L instead of DiffNeRV | video regression · UVG | PSNR: +0.42 dB | Helped |
| use PNeRV instead of DiffNeRV | video regression · UVG | LPIPS: 0.176 -> 0.163 | Helped |
| use PNeRV-L instead of DiffNeRV | video regression · DAVIS | PSNR: +0.88 dB | Helped |
| use PNeRV instead of HNeRV | video regression · DAVIS | LPIPS: 0.321 -> 0.285 | Helped |
| add pyramidal shortcuts with BSM to serial HNeRV (ablation) | PNeRV-M · Parkour | PSNR: 25.07 -> 24.81; 26.56 -> 26.02; 24.34 -> 27.13 | Mixed |
| add pyramidal shortcuts with BSM to serial HNeRV (ablation) | PNeRV-M · HoneyB | PSNR: 36.65 -> 40.34; 36.72 -> 41.36; 38.96 -> 42.59 | Helped |
| replace KFC with bilinear upsampling while retaining BSM (ablation) | PNeRV · DAVIS (7 videos) | PSNR: 31.30 -> 29.08 | Hurt |
| replace KFC with Deconv while retaining BSM (ablation) | PNeRV · DAVIS (7 videos) | PSNR: 31.30 -> 29.96 | Hurt |
| replace BSM with concatenation while retaining KFC (ablation) | PNeRV · DAVIS (7 videos) | PSNR: 31.30 -> 28.68 | Hurt |
| replace BSM with GRU while retaining KFC (ablation) | PNeRV · DAVIS (7 videos) | PSNR: 31.30 -> 29.31 | Hurt |
| replace BSM with LSTM while retaining KFC (ablation) | PNeRV · DAVIS (7 videos) | PSNR: 31.30 -> 29.04 | Hurt |
Limitations
- Hierarchical structure brings higher computational complexity.
- Theoretical analysis relies on an estimated upper bound and an informal rate-of-dynamics quantity that is hard to compute.
Experimental setup
Datasets: UVG DAVIS
Benchmarks: UVG DAVIS
Metrics: PSNR MS-SSIM LPIPS FVD
Baselines: NeRV E-NeRV HNeRV DiffNeRV DivNeRV
Builds on: NeRV HNeRV DiffNeRV PixelShuffle FPN
Closest papers by meaning
- MetaNeRV: Meta Neural Representations for Videos with Spatial-Temporal Guidance (AAAI 2025 · similarity 0.87)
- Combining Frame and GOP Embeddings for Neural Video Representation (CVPR 2024 · similarity 0.87)
- CoANeRV: Coordinate-Aware Token-Space Neural Video Representation (NeurIPS 2026 · similarity 0.87)
- Boosting Neural Representations for Videos with a Conditional Decoder (CVPR 2024 · similarity 0.86)
- DS-NeRV: Implicit Neural Video Representation with Decomposed Static and Dynamic Codes (CVPR 2024 · similarity 0.85)
- Bias for Action: Video Implicit Neural Representations with Bias Modulation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.