Research map / Video / Video segmentation and tracking
CoANeRV: Coordinate-Aware Token-Space Neural Video Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CoANeRV, a feed-forward token-space neural video representation that stores video content in tokens and reconstructs spatio-temporal coordinates with a shared cross-attention decoder, using axis-adaptive encoding, temperature-scaled attention, and block-wise querying.
Problem
Existing NeRV-style video representations need costly per-video optimization or video-specific weight generation, which limits scalable amortized video representation.
Approach
Shift video-specificity from decoder weights to compact feed-forward video tokens read by a shared coordinate-conditioned cross-attention decoder, with anisotropic (x,y,t) positional encoding, temperature-scaled attention via fused SDPA, and spatial tiling of coordinate queries to bound peak memory.
Key results
- At 4 frames, CoANeRV reaches 34.8 dB PSNR on K400 (150 epochs) versus 26.6 dB for FastNeRV; 20-epoch checkpoint already gets 31.5 dB.
- Versus video-adapted ANR-V at 8 frames large size, CoANeRV-L uses 4.0 GB peak memory vs 74 GB for ANR-V-L, with higher K400 PSNR (34.9 vs 32.0 dB).
- Quantized token storage at ~20 KB gives 32.51 dB for CoANeRV versus 28.3 dB for 7-bit FastNeRV, but H.264 and AV1 remain competitive and decoding is slower (7.7 fps at 1080p).
Surprising findings
- Decoder reconstruction at 1080p runs at only 7.7 fps versus 112.8 fps for DCVC-RT, despite token extraction at 247 fps.
- Conventional codecs H.264 and AV1 remain competitive with token-space storage at comparable sizes and decode faster.
Limitations
- Coordinate-attention decoder has limited throughput (7.7 fps at 1080p).
- Focus on short clips (4/8/16 frames) at 256x256 resolution.
- Storage study uses only post-training quantization and Huffman coding, without QAT, learned entropy model, or rate control.
Experimental setup
Datasets: Kinetics-400 (10k subset)
Benchmarks: Kinetics-400 Something-Something V2 UCF101
Metrics: PSNR SSIM MS-SSIM LPIPS peak memory GPU hours
Baselines: TransINR GINR FastNeRV ANR-V NeRV H.264 AV1 DCVC-RT
Builds on: NeRV ANR FastNeRV LAG-INR hypernetworks
Closest papers by meaning
- Efficient Long Video Tokenization via Coordinate-based Patch Reconstruction (CVPR 2025 · similarity 0.88)
- MetaNeRV: Meta Neural Representations for Videos with Spatial-Temporal Guidance (AAAI 2025 · similarity 0.87)
- PNeRV: Enhancing Spatial Consistency via Pyramidal Neural Representation for Videos (CVPR 2024 · similarity 0.87)
- DS-NeRV: Implicit Neural Video Representation with Decomposed Static and Dynamic Codes (CVPR 2024 · similarity 0.86)
- Boosting Neural Representations for Videos with a Conditional Decoder (CVPR 2024 · similarity 0.86)
- Combining Frame and GOP Embeddings for Neural Video Representation (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.