Research map / Video / Video segmentation and tracking
Direction-Aware Video Demoiréing with Temporal-Guided Bilateral Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DTNet, a two-stage video demoiréing network combining direction-aware DCT-domain moiré removal with multi-frame alignment and temporal-guided bilateral (affine grid) color/detail refinement.
Problem
Video demoiréing: removing moiré patterns from videos captured of screens while restoring color and fine details and keeping temporal consistency; existing methods ignore the directional, formation-process characteristics of moiré.
Approach
Removes moiré in the block-DCT domain using eight directional DCT modes and a learnable band-reject filter, aligns neighboring frames with EDVR-style alignment plus deformable convolution, then applies a temporal-guided bilateral grid (affine coefficients looked up with guidance maps from adjacent aligned frames) for color and detail refinement.
Key results
- On TCL-V1, DTNet reaches 24.119 dB PSNR, a +2.093 dB gain over the best prior method (ESDNet, 22.026 dB).
- On iPhone-V1, DTNet-full achieves 24.821 dB PSNR, 2.284 dB above ESDNet, with DTNet-light at 0.233 T MACs.
- Ablation on TCL-V2: removing alignment drops PSNR from 26.153 to 23.635 dB, the largest drop among components.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add directional DCT (ablation) | DTNet · TCL-V2 | PSNR: 26.005 -> 26.153 | Helped |
| add directional DCT (ablation) | DTNet · TCL-V2 | SSIM: 0.872 -> 0.877 | Helped |
| add directional DCT (ablation) | DTNet · TCL-V2 | LPIPS: 0.140 -> 0.128 | Helped |
| use multi-scale alignment and deformable convolution instead of concatenation and two convolutional layers (ablation) | DTNet · TCL-V2 | PSNR: 23.635 -> 26.153 | Helped |
| use multi-scale alignment and deformable convolution instead of concatenation and two convolutional layers (ablation) | DTNet · TCL-V2 | SSIM: 0.848 -> 0.877 | Helped |
| use multi-scale alignment and deformable convolution instead of concatenation and two convolutional layers (ablation) | DTNet · TCL-V2 | LPIPS: 0.196 -> 0.128 | Helped |
| add temporal-guided bilateral learning instead of concatenation and two convolutional layers (ablation) | DTNet · TCL-V2 | PSNR: 25.813 -> 26.153 | Helped |
| add temporal-guided bilateral learning instead of concatenation and two convolutional layers (ablation) | DTNet · TCL-V2 | SSIM: 0.871 -> 0.877 | Helped |
| add temporal-guided bilateral learning instead of concatenation and two convolutional layers (ablation) | DTNet · TCL-V2 | LPIPS: 0.144 -> 0.128 | Helped |
| use multi-frame input rather than repeated copies of a single frame (ablation) | DTNet · TCL-V2 | PSNR: 25.914 -> 26.153 | Helped |
| use multi-frame input rather than repeated copies of a single frame (ablation) | DTNet · TCL-V2 | SSIM: 0.870 -> 0.877 | Helped |
| use multi-frame input rather than repeated copies of a single frame (ablation) | DTNet · TCL-V2 | LPIPS: 0.145 -> 0.128 | Helped |
Surprising findings
- Removing the directional DCT module reduces PSNR by only about 0.15 dB, while removing temporal-guided bilateral learning costs 0.34 dB, so the alignment module contributes far more than either novel component.
Limitations
- Evaluated only on the VDmoiré dataset with three-frame input, so generalization to other video settings is untested.
- Heavier parameter count in DTNet-full (7.36M) than DMCNN, though lower MACs than WDNet and ESDNet.
Experimental setup
Datasets: VDmoiré
Benchmarks: VDmoiré TCL-V1 VDmoiré TCL-V2 VDmoiré iPhone-V1 VDmoiré iPhone-V2
Metrics: PSNR SSIM LPIPS FVD FSIM Params MACs
Baselines: DMCNN WDNet ESDNet VDmoiré FPANet
Builds on: EDVR MBCNN bilateral grid learning H.264 directional intra prediction
Closest papers by meaning
- UniDemoiré: Towards Universal Image Demoiréing with Data Generation and Synthesis (AAAI 2025 · similarity 0.85)
- Learning Image Demoiréing from Unpaired Real Data (AAAI 2024 · similarity 0.84)
- One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution (NeurIPS 2025 · similarity 0.84)
- Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring (CVPR 2024 · similarity 0.84)
- TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Models (CVPR 2024 · similarity 0.83)
- Motion Consistency Model: Accelerating Video Diffusion with Disentangled Motion-Appearance Distillation (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.