Research map / Video / Video segmentation and tracking
FLAVC: Learned Video Compression with Feature Level Attention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FLAVC, a transformer-based learned video compression model whose feature-level attention (FLA), dense overlapping patcher, and transformer-CNN mixed encoder replace motion-based context modeling, achieving state-of-the-art rate-distortion versus HM-16.26 and prior LVC methods.
Problem
Learned video compression relies on motion estimation/compensation and CNN-based context models, which limit generalizability and global perception, while existing transformer-based entropy models (VCT) are memory-costly and suffer from a single-scale feature bottleneck.
Approach
Feature-level attention converts local patch embeddings into batch-wise one-dimensional vectors and replaces the attention weight matrix with a global context matrix (k^T v), enabling full-frame perception without motion signals; it is combined with a dense overlapping patcher and a Transformer-CNN mixed encoder to reduce the feature bottleneck.
Key results
- On UVG, FLAVC achieves -61.46% BD-rate (PSNR) versus HM-16.26, compared with -10.02% for DCVC-TCM and 12.57% for VCT.
- On MCL-JCV, FLAVC reaches -63.76% BD-rate (PSNR) and -80.15% (MS-SSIM) versus HM-16.26.
- FLAVC-Small reduces encoding cost to 1092.82 kMACs/pixel versus 1570.6 for VCT while keeping strong RD performance (UVG PSNR BD-rate -41.94%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace standard attention with feature-level attention (FLA) (ablation) | VCT-based entropy model · HEVC-Class B | bit cost | Helped |
| add Transformer-CNN mixed encoder (TCM) (ablation) | VCT baseline · HEVC-Class B | rate-distortion performance | Hurt |
| add dense overlapping patcher (DP) while retaining ELIC encoder (ablation) | VCT baseline · HEVC-Class B | compression performance | Hurt |
| add dense overlapping patcher (DP) with TCM encoder (ablation) | FLA+TCM · HEVC-Class B | compression performance | Helped |
| reduce FLA expansion from 8× to 4× (ablation) | FLAVC · UVG | BD-rate (PSNR): -61.46% -> -41.94% | Hurt |
Surprising findings
- The model saturates earlier at high bitrates on lower-resolution videos (HEVC Class C), attributed to the encoder's information bottleneck.
- The dense patcher alone with the ELIC encoder decreased compression performance; it only helped when combined with the TCM encoder.
Limitations
- Earlier saturation in high bit-rate regime, especially for low-resolution videos, due to the feature bottleneck.
- Background noise in transformer-based entropy model outputs.
- Higher parameter count (143M for FLAVC) than several CNN-based baselines.
Experimental setup
Datasets: Vimeo-90K
Benchmarks: UVG MCL-JCV HEVC Class B HEVC Class C
Metrics: BD-rate PSNR MS-SSIM kMACs per pixel parameters
Baselines: VCT DCVC-TCM DCVC-DC ST-XCT DHVC CANF-VC CANF-VC++ SDD MCRT HM-16.26 VTM-13.2
Builds on: VCT Efficient Attention TCM ELIC DCVC-TCM
Closest papers by meaning
- FVC: A New Framework Towards Deep Video Compression in Feature Space (CVPR 2021 · similarity 0.88)
- Linear Attention Modeling for Learned Image Compression (CVPR 2025 · similarity 0.87)
- Neural Video Compression with Feature Modulation (CVPR 2024 · similarity 0.86)
- One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding (NeurIPS 2025 · similarity 0.86)
- GLIC: General Format Learned Image Compression (AAAI 2025 · similarity 0.85)
- ECVC: Exploiting Non-Local Correlations in Multiple Frames for Contextual Video Compression (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.