Research map / Video / Video segmentation and tracking
MetaNeRV: Meta Neural Representations for Videos with Spatial-Temporal Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
MetaNeRV meta-learns (MAML) an initialization for NeRV-style image-wise video representations, with multi-resolution loss and progressive frame-count training, enabling fast fitting of unseen videos.
Problem
NeRV-based video representations must be trained from scratch for each new video, which is slow when adapting to many diverse videos.
Approach
Use MAML to learn an initialization for an E-NeRV-style frame-index-to-image network so few gradient steps suffice for unseen videos; add multi-resolution auxiliary heads (spatial guidance) and progressive increase of frames per inner task (temporal guidance).
Key results
- On EchoNet-LVH, MetaNeRV reaches 24.05 PSNR after one inner step vs 6.36 for E-NeRV (three-step: 26.94 vs 16.44).
- On HMDB-51 one-step PSNR is 18.43 for MetaNeRV vs 11.13 for E-NeRV; the paper reports roughly 9x speedup in convergence.
- With quantization-aware training and entropy coding, MetaNeRV is reported to outperform H.264 and HEVC and perform comparably to state-of-the-art learned codecs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace random initialization with meta-learned initialization | NeRV-based video representation · MCL_JCV | PSNR (Step1/Step3): 11.13/15.78 -> 17.60/22.02 | Helped |
| replace random initialization with meta-learned initialization | NeRV-based video representation · HMDB-51 | PSNR (Step1/Step3): 11.13/15.78 -> 18.43/21.43 | Helped |
| replace random initialization with meta-learned initialization | NeRV-based video representation · UCF101 | PSNR (Step1/Step3): 10.04/15.04 -> 18.69/22.46 | Helped |
| replace random initialization with meta-learned initialization | NeRV-based video representation · EchoNet-LVH | PSNR (Step1/Step3): 6.36/16.44 -> 24.05/26.94 | Helped |
| replace random initialization with meta-learned initialization | NeRV-based video representation · EchoCP | PSNR (Step1/Step3): 7.79/18.53 -> 23.34/25.44 | Helped |
| add temporal guidance (progressive frame learning) (ablation) | MetaNeRV · HMDB-51 | PSNR (Step1/Step3): 16.89/19.43 -> 17.3/19.31 | Mixed |
| add temporal guidance (progressive frame learning) (ablation) | MetaNeRV · UCF101 | PSNR (Step1/Step3): 16.4/19.08 -> 16.83/19.53 | Helped |
| add temporal guidance (progressive frame learning) (ablation) | MetaNeRV · EchoNet-LVH | PSNR (Step1/Step3): 21.66/23.72 -> 22.16/24.31 | Helped |
| add temporal guidance (progressive frame learning) (ablation) | MetaNeRV · EchoCP | PSNR (Step1/Step3): 19.77/21.78 -> 20.63/23.15 | Helped |
| add spatial guidance (multi-resolution loss) (ablation) | MetaNeRV · HMDB-51 | PSNR (Step1/Step3): 16.89/19.43 -> 17.41/22.06 | Helped |
| add spatial guidance (multi-resolution loss) (ablation) | MetaNeRV · UCF101 | PSNR (Step1/Step3): 16.4/19.08 -> 18.04/21.96 | Helped |
| add spatial guidance (multi-resolution loss) (ablation) | MetaNeRV · EchoNet-LVH | PSNR (Step1/Step3): 21.66/23.72 -> 23.19/25.63 | Helped |
Surprising findings
- Initialization trained on one real-world dataset (HMDB-51) transfers to MCL_JCV and to ultrasound datasets, outperforming random initialization.
Limitations
- Cannot represent videos at higher resolutions without retraining.
- May face convergence challenges with limited video training data.
Experimental setup
Datasets: UCF101 HMDB-51 MCL_JCV EchoCP EchoNet-LVH Bunny UVG HOLLYWOOD2 SVW OOPS
Benchmarks: UCF101 HMDB-51 MCL_JCV EchoCP EchoNet-LVH HOLLYWOOD2 SVW OOPS Bunny UVG
Metrics: PSNR MS-SSIM BPP training time
Baselines: NeRV E-NeRV FFNeRV HNeRV H.264 HEVC HLVC PS-NeRV
Builds on: NeRV E-NeRV MAML
Closest papers by meaning
- Combining Frame and GOP Embeddings for Neural Video Representation (CVPR 2024 · similarity 0.87)
- PNeRV: Enhancing Spatial Consistency via Pyramidal Neural Representation for Videos (CVPR 2024 · similarity 0.87)
- CoANeRV: Coordinate-Aware Token-Space Neural Video Representation (NeurIPS 2026 · similarity 0.87)
- DS-NeRV: Implicit Neural Video Representation with Decomposed Static and Dynamic Codes (CVPR 2024 · similarity 0.87)
- Boosting Neural Representations for Videos with a Conditional Decoder (CVPR 2024 · similarity 0.85)
- HyperNVD: Accelerating Neural Video Decomposition via Hypernetworks (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.