Research map / Video / Video segmentation and tracking
TTA-EVF: Test-Time Adaptation for Event-based Video Frame Interpolation via Reliable Pixel and Sample Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TTA-EVF, an online test-time adaptation method for event-based video frame interpolation that uses teacher-student reliable pixel sampling and patch-mixed memory-bank sampling, and releases the ERDS dataset for continuous distribution shifts.
Problem
Event-based video frame interpolation degrades when applied to target domains whose event distributions differ from training (camera settings, illumination), and no high-frame-rate ground truth exists in the target domain.
Approach
Adapt a pre-trained event-based VFI network online using only low-frame-rate target videos: reliable pixels are chosen by consistency between a student and a momentum teacher with a scene-adaptive threshold, and a memory bank of reliable samples is patch-mixed with current frames to prevent overfitting. A Norm-Residual block with group normalization is inserted into encoders so that only part of t
Key results
- Cross-domain TTA-EVF with TimeLens reaches 34.24 dB PSNR on B->R versus 32.04 dB without adaptation and 33.01 dB for the Cycle baseline.
- On HighREV with TimeLens, TTA-EVF is 1.43 dB above supervised training on the target domain.
- On ERDS continuous shift with CBMNet, TTA-EVF averages 33.53 dB PSNR versus 31.42 dB without adaptation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply TTA-EVF instead of Cycle | TimeLens · BS-ERGB to HighREV | PSNR: 24.11 -> 26.91 | Helped |
| apply TTA-EVF instead of Cycle | TimeLens · BS-ERGB to HS-ERGB | PSNR: 30.66 -> 32.13 | Helped |
| apply TTA-EVF instead of Cycle | TimeReplayer · ERDS total | PSNR: 32.99 -> 33.71 | Helped |
| add Reliable Pixel Sampling and Patch-Mixed Sampling to self-supervised adaptation with NR Block (ablation) | TimeLens · Cross-domain (BS-ERGB to HS-ERGB) | PSNR: 31.53 -> 32.13 | Helped |
| add Patch-Mixed Sampling (ablation) | TimeLens · Cross-domain (BS-ERGB to HS-ERGB) | PSNR: 31.84 -> 32.13 | Helped |
| add Reliable Pixel Sampling (ablation) | TimeLens · Cross-domain (BS-ERGB to HS-ERGB) | PSNR: 31.92 -> 32.13 | Helped |
| use GN in the NR Block instead of BN (ablation) | TimeLens · Cross-domain (HS-ERGB) | PSNR: 31.57 -> 32.13 | Helped |
| replace GN with LN in the NR Block (ablation) | TimeLens · Cross-domain (HS-ERGB) | PSNR: 32.13 -> 31.94 | Hurt |
| use an adaptive reliability threshold instead of fixed threshold 0.05 (ablation) | TimeLens with RPS · Cross-domain (HS-ERGB) | PSNR: 31.85 -> 32.13 | Helped |
| set PMS mixing-ratio minimum to 0.7 rather than 0.5, 0.6, 0.8, or 0.9 (ablation) | TTA-EVF · Cross-domain | PSNR: 31.92, 32.06, 32.13, 32.04, 31.93 | Mixed |
| update every 5 intervals instead of every interval (ablation) | TimeLens with TTA-EVF · HS-close | PSNR: 32.11 -> 31.44 | Hurt |
Surprising findings
- Updating full parameters or BatchNorm at test time degrades VFI performance below no adaptation in the ablation, and the plain self-supervised cycle loss yields little gain.
Limitations
- Adaptive threshold and reliability check rely on hyperparameters such as eta=0.9 and minimum motion m=1.
- TimeReplayer was re-implemented because public code was unavailable.
- Evaluated only on three real event datasets and one new continuous-shift dataset.
Experimental setup
Datasets: BS-ERGB HS-ERGB HighREV ERDS
Benchmarks: BS-ERGB HS-ERGB HighREV ERDS
Metrics: PSNR SSIM
Baselines: No Adaptation MetaVFI (Meta) Cycle (TimeReplayer self-supervised) Supervised training on target Unsupervised TimeReplayer
Builds on: TimeLens CBMNet TimeReplayer MetaVFI Mean Teacher
Closest papers by meaning
- Repurposing Pre-trained Video Diffusion Models for Event-based Video Interpolation (CVPR 2025 · similarity 0.91)
- Video Frame Interpolation via Direct Synthesis with the Event-based Reference (CVPR 2024 · similarity 0.90)
- TimeTracker: Event-based Continuous Point Tracking for Video Frame Interpolation with Non-linear Motion (CVPR 2025 · similarity 0.87)
- Video Frame Prediction from a Single Image and Events (AAAI 2024 · similarity 0.85)
- E2HQV: High-Quality Video Generation from Event Camera via Theory-Inspired Model-Aided Deep Learning (AAAI 2024 · similarity 0.85)
- EF-3DGS: Event-Aided Free-Trajectory 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.