Research map / Video / Video segmentation and tracking
Event-based video reconstruction
Reconstructs high-speed, high-dynamic-range video from event cameras with learned losses and simulation.
63 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2019–2026) · an approach in Video segmentation and tracking, keyword cluster “interpolation · flow · event”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 1 of 2 tracked benchmarks.
15 papers in 2023–24 and 28 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 3.4% to 5.2%.
Open questions the data points to
- GoPro (PSNR): best went from 36.7 (up to 2024) to 37.13 in 2025 — TimeTracker: Event-based Continuous Point Tracking for Video Frame Interpolation with Non-linear Motion (CVPR 2025 · 2 citations)
- Papers report ET-Net on HQF (SSIM) anywhere from 0.534 to 0.658: comparisons against it depend on whose number you take.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2019 · Events-To-Video: Bringing Modern Computer Vision to Event Cameras (CVPR 2019 · 428 citations)
- 2020 · Video to Events: Recycling Video Datasets for Event Cameras (CVPR 2020 · 235 citations)
- 2021 · Time Lens: Event-Based Video Frame Interpolation (CVPR 2021 · 227 citations)
- 2022 · Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation (AAAI 2022 · 65 citations)
- 2023 · Frame-Event Alignment and Fusion Network for High Frame Rate Tracking (CVPR 2023 · 71 citations)
- 2024 · HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensors (AAAI 2024 · 48 citations)
- 2025 · ETAP: Event-based Tracking of Any Point (CVPR 2025 · 6 citations)
- 2026 · BAT: Learning Event-based Optical Flow with Bidirectional Adaptive Temporal Correlation (AAAI 2026 · 1 citation)
Earliest papers here
- Events-To-Video: Bringing Modern Computer Vision to Event Cameras (CVPR 2019 · 428 citations)
- Bringing a Blurry Frame Alive at High Frame-Rate With an Event Camera (CVPR 2019 · 304 citations)
- Event-Based High Dynamic Range Image and Very High Frame Rate Video Generation Using Conditional Generative Adversarial Networks (CVPR 2019 · 191 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
GoPro · PSNR (higher is better)
4 papers, 2020–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 31.79 | Learning Event-Based Motion Deblurring (CVPR 2020 · 198 citations) |
| 2024 | 36.7 | Frequency-aware Event-based Video Deblurring for Real-World Motion Blur (CVPR 2024 · 32 citations) |
| 2025 | 37.13 | TimeTracker: Event-based Continuous Point Tracking for Video Frame Interpolation with Non-linear Motion (CVPR 2025 · 2 citations) |
GoPro · SSIM (higher is better)
4 papers, 2020–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 0.949 | Learning Event-Based Motion Deblurring (CVPR 2020 · 198 citations) |
| 2024 | 0.981 | Motion Deblurring via Spatial-Temporal Collaboration of Frames and Events (AAAI 2024 · 15 citations) |
What has been tried
43 papers here report 463 measured changes (a component added, removed or swapped, compared with a baseline): helped 82%, no clear effect 4%, hurt 12%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add TemporalFormer (TF) | N-Caltech101 · top-1 accuracy | 72.14 -> 81.54 | Helped | HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensors (AAAI 2024 · 48 citations) |
| replace raw events with ETS-processed events without LETC or NIAM | RLED · MSE | 0.019 -> 0.045 | Hurt | Seeing Motion at Nighttime with an Event Camera (CVPR 2024 · 33 citations) |
| Add FCFE | GoPro · PSNR | 31.14 -> 33.39 | Helped | Frequency-aware Event-based Video Deblurring for Real-World Motion Blur (CVPR 2024 · 32 citations) |
| Add REFE to MAFG | IJRR · MSE | 0.080 -> 0.051 | Helped | E2HQV: High-Quality Video Generation from Event Camera via Theory-Inspired Model-Aided Deep Learning (AAAI 2024 · 18 citations) |
| Add MSC | GoPro · PSNR | 33.40 -> 35.93 | Helped | Motion Deblurring via Spatial-Temporal Collaboration of Frames and Events (AAAI 2024 · 15 citations) |
| Add proposal augmentation | Penguin ED test set · mAP (average) | 0.17 -> 0.58 | Helped | Low-power Continuous Remote Behavioral Localization with Event Cameras (CVPR 2024 · 14 citations) |
| Add feature-alignment loss | EVIMO2 · AJ | 0.655 -> 0.661 | Helped | ETAP: Event-based Tracking of Any Point (CVPR 2025 · 6 citations) |
| add common fusion instead of direct fusion | Fast-DSEC · EPE | 0.84 -> 0.58 | Helped | Bridge Frame and Event: Common Spatiotemporal Fusion for High-Dynamic Scene Optical Flow (CVPR 2025 · 3 citations) |
| Add convolutional integration instead of pixel-wise integration | 96 test scenes · PSNR | 5.755 -> 11.471 | Helped | Unified Reconstruction of Static and Dynamic Scenes from Events (CVPR 2025 · 3 citations) |
| Use the first and last intensity frames rather than only the first frame | DriveGaze E4-S3 · WAR | 88.6 -> 91.2 | Helped | DriveGazen: Event-Based Driving Status Recognition Using Conventional Camera (AAAI 2025 · 2 citations) |
| Increase the λ range from 0.0–0.0 to 0.2–0.6 | N-Caltech101 · accuracy | -0.23 | Hurt | EventZoom: A Progressive Approach to Event-Based Data Augmentation for Enhanced Neuromorphic Vision (AAAI 2025 · 2 citations) |
| Add superpixel segmentation | GoPro · PSNR | 30.62 -> 37.47 | Helped | TimeTracker: Event-based Continuous Point Tracking for Video Frame Interpolation with Non-linear Motion (CVPR 2025 · 2 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| ET-Net | HQF | SSIM | 0.534 – 0.658 | 2 papers, e.g. NeurIPS 2024 and AAAI 2024 |
What reviewers will expect
Datasets and baselines named in the 12 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- N-Caltech101 · 2 of 12 papers
- MultiFlow · 2 of 12 papers
Baselines
- E2VID · 3 of 12 papers
- ET-Net · 2 of 12 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 43 papers; from OpenAlex and the papers' affiliations).
- Peking University · 6 papers
- National Institute of Informatics · 3 papers
- Huazhong University of Science and Technology · 3 papers
- Nanyang Technological University · 3 papers
- Hong Kong University of Science and Technology · 3 papers
- University of Chinese Academy of Sciences · 2 papers
- Beijing Institute of Technology · 2 papers
- Dalian University of Technology · 2 papers
- Southeast University · 2 papers
- Beijing University of Technology · 2 papers
Use it
Download BibTeX of 39 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Learned optical flow estimation (73 papers)
- Video frame interpolation (60 papers)
- Spike and neuromorphic deblurring (57 papers)
Elsewhere in the topic
- Self-supervised video representation learning (167 papers)
- Video restoration with temporal priors (159 papers)
- Layered video decomposition and dynamics (140 papers)
- Stochastic video prediction (125 papers)
- Physics-inspired motion prediction (113 papers)
- Sign language and physiological video analysis (99 papers)
- Weakly supervised video anomaly detection (99 papers)
- Spatio-temporal predictive modeling (91 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).