Research map / Video / Video segmentation and tracking
Video restoration with temporal priors
Enhances and restores degraded video using internal priors, state-space models and temporal coherence.
159 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Video segmentation and tracking, keyword cluster “detection · anomaly · supervised”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 4 of 6 tracked benchmarks.
63 papers in 2023–24 and 43 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 14.3% to 8.0%.
Open questions the data points to
- LIVE-VQC (SRCC): best went from 0.834 (up to 2024) to 0.895 in 2025 — FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 · 29 citations)
- KoNViD-1k (SRCC): best went from 0.857 (up to 2024) to 0.915 in 2025 — FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 · 29 citations)
- LIVE-VQC (PLCC): best went from 0.849 (up to 2024) to 0.895 in 2025 — FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 · 29 citations)
- LSVQ1080p (SRCC): best went from 0.782 (up to 2024) to 0.828 in 2025 — FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 · 29 citations)
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Highway Vehicle Counting in Compressed Domain (CVPR 2016 · 32 citations)
- 2017 · Video Desnowing and Deraining Based on Matrix Decomposition (CVPR 2017 · 195 citations)
- 2018 · HSA-RNN: Hierarchical Structure-Adaptive RNN for Video Summarization (CVPR 2018 · 243 citations)
- 2019 · Deep Video Inpainting (CVPR 2019 · 217 citations)
- 2020 · Supervised Raw Video Denoising With a Benchmark Dataset on Dynamic Scenes (CVPR 2020 · 118 citations)
- 2021 · Patch-VQ: 'Patching Up' the Video Quality Problem (CVPR 2021 · 197 citations)
- 2022 · Bringing Old Films Back to Life (CVPR 2022 · 63 citations)
- 2023 · A Simple Baseline for Video Restoration With Grouped Spatial-Temporal Shift (CVPR 2023 · 73 citations)
- 2024 · WaveFormer: Wavelet Transformer for Noise-Robust Video Inpainting (AAAI 2024 · 55 citations)
- 2025 · FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 · 29 citations)
- 2026 · DLVINet: Advancing Dual-Lens Video Inpainting Beyond Parallax Constraints (AAAI 2026 · 2 citations)
Earliest papers here
- Highway Vehicle Counting in Compressed Domain (CVPR 2016 · 32 citations)
- Video Desnowing and Deraining Based on Matrix Decomposition (CVPR 2017 · 195 citations)
- A Novel Tensor-Based Video Rain Streaks Removal Approach via Utilizing Discriminatively Intrinsic Priors (CVPR 2017 · 181 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
LIVE-VQC · SRCC (higher is better)
6 papers, 2021–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 0.77 | Patch-VQ: 'Patching Up' the Video Quality Problem (CVPR 2021 · 197 citations) |
| 2023 | 0.814 | MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videos (CVPR 2023 · 69 citations) |
| 2024 | 0.834 | Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment (AAAI 2024 · 13 citations) |
| 2025 | 0.895 | FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 · 29 citations) |
KoNViD-1k · SRCC (higher is better)
5 papers, 2021–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 0.791 | Patch-VQ: 'Patching Up' the Video Quality Problem (CVPR 2021 · 197 citations) |
| 2023 | 0.851 | MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videos (CVPR 2023 · 69 citations) |
| 2024 | 0.857 | PTM-VQA: Efficient Video Quality Assessment Leveraging Diverse PreTrained Models from the Wild (CVPR 2024 · 12 citations) |
| 2025 | 0.915 | FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 · 29 citations) |
DAVIS · PSNR (higher is better)
4 papers, 2022–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 34.22 | DLFormer: Discrete Latent Transformer for Video Inpainting (CVPR 2022 · 43 citations) |
DAVIS · SSIM (higher is better)
4 papers, 2022–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 0.977 | DLFormer: Discrete Latent Transformer for Video Inpainting (CVPR 2022 · 43 citations) |
LIVE-VQC · PLCC (higher is better)
4 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 0.839 | MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videos (CVPR 2023 · 69 citations) |
| 2024 | 0.849 | Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment (AAAI 2024 · 13 citations) |
| 2025 | 0.895 | FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 · 29 citations) |
LSVQ1080p · SRCC (higher is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 0.782 | Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment (AAAI 2024 · 13 citations) |
| 2025 | 0.828 | FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 · 29 citations) |
What has been tried
139 papers here report 1,453 measured changes (a component added, removed or swapped, compared with a baseline): helped 76%, no clear effect 5%, hurt 17%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add DWT-based frequency separation to attention computation | Video inpainting evaluation (dataset not stated) · PSNR | 31.326 -> 33.264 | Helped | WaveFormer: Wavelet Transformer for Noise-Robust Video Inpainting (AAAI 2024 · 55 citations) |
| Add AE-Net to VE-Net, with FEGM and FRGM replaced by traditional convolutions | UVEB test set · PSNR | 24.41 -> 25.15 | Helped | UVEB: A Large-scale Benchmark and Baseline Towards Real-World Underwater Video Enhancement (CVPR 2024 · 33 citations) |
| Add the dynamic filter in Phase 3 | Synthetic-I · RMSE | 0.0747 -> 0.0594 | Helped | EulerMormer: Robust Eulerian Motion Magnification via Dynamic Filtering within Transformer (AAAI 2024 · 32 citations) |
| Add the spatial rectifier | BVI-SR · SRCC / PLCC | 0.448 / 0.554 -> 0.837 / 0.891 | Helped | Modular Blind Video Quality Assessment (CVPR 2024 · 32 citations) |
| add motion features to spatial features | FineVD; LIVE-VQC; YouTube-UGC · SRCC | 0.8210 -> 0.8264; 0.7832 -> 0.8061; 0.8084 -> 0.8349 | Helped | FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 · 29 citations) |
| add NRFM instead of randomly matching clear reference frames | GoProHazy · FADE | 0.9204 -> 0.7598 | Helped | Driving-Video Dehazing with Non-Aligned Regularization for Safety Assistance (CVPR 2024 · 29 citations) |
| use attentiveness scores for token pruning instead of saliency-aware merging | MSRVTT · Mean retrieval score | 50.0 -> 49.0 | Hurt | vid-TLDR: Training Free Token Merging for Light-weight Video Transformer (CVPR 2024 · 27 citations) |
| Add GRUM | ATSyn-dynamic · PSNR | 28.87 -> 29.06 | Helped | Spatio-Temporal Turbulence Mitigation: A Translational Perspective (CVPR 2024 · 25 citations) |
| add the diffusion process to vanilla NAFNet | Average (Rain, Haze, Snow) · PSNR | 26.83 -> 28.52 | Helped | Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal (CVPR 2024 · 22 citations) |
| Add middle and deep frequency bands (three-level architecture) | Synthetic Dataset · SSIM | 0.8582 -> 0.9104 | Helped | Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecture (CVPR 2024 · 20 citations) |
| add directional DCT | TCL-V2 · PSNR | 26.005 -> 26.153 | Helped | Direction-Aware Video Demoiréing with Temporal-Guided Bilateral Learning (AAAI 2024 · 14 citations) |
| add mean(|·|) statistics to DACA | LLRVD · PSNR | 36.95 -> 37.02 | Helped | Binarized Low-light Raw Video Enhancement (CVPR 2024 · 14 citations) |
What reviewers will expect
Datasets and baselines named in the 24 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- VFHQ · 4 of 24 papers
- DAVIS · 4 of 24 papers
- Vimeo-90K · 2 of 24 papers
- KoNViD-1k · 2 of 24 papers
- YouTube-UGC · 2 of 24 papers
- LIVE-VQC · 2 of 24 papers
- LSVQ · 2 of 24 papers
- Dynamic Replica · 2 of 24 papers
Baselines
- CodeFormer · 4 of 24 papers
- BasicVSR++ · 3 of 24 papers
- VRT · 3 of 24 papers
- RVRT · 3 of 24 papers
- PGTFormer · 3 of 24 papers
- RestoreFormer · 2 of 24 papers
- DR2 · 2 of 24 papers
- DiffBIR · 2 of 24 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 106 papers; from OpenAlex and the papers' affiliations).
- City University of Hong Kong · 7 papers
- Nanyang Technological University · 5 papers
- Nanjing University of Science and Technology · 5 papers
- Tsinghua University · 5 papers
- Hong Kong University of Science and Technology · 5 papers
- Zhejiang University · 4 papers
- Westlake University · 4 papers
- Beijing Institute of Technology · 4 papers
- The University of Hong Kong · 3 papers
- University of Chinese Academy of Sciences · 3 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Self-supervised video representation learning (167 papers)
- Sign language and physiological video analysis (99 papers)
- Weakly supervised video anomaly detection (99 papers)
- Spatio-temporal predictive modeling (91 papers)
- Deepfake video detection (47 papers)
Elsewhere in the topic
- Layered video decomposition and dynamics (140 papers)
- Stochastic video prediction (125 papers)
- Physics-inspired motion prediction (113 papers)
- Matching-based video object segmentation (86 papers)
- Learned optical flow estimation (73 papers)
- Learned video compression (68 papers)
- Mask propagation for unsupervised video segmentation (67 papers)
- Video semantic and panoptic segmentation (67 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).