Research map / Video / Video generation
SeeClear: Semantic Distillation Enhances Pixel Condensation for Video Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
placeholder
Problem
Diffusion-based video super-resolution produces realistic frames but suffers from inconsistent details across frames because stochastic sampling disrupts pixel-level inter-frame alignment.
Approach
Replace pixel-level alignment with semantic alignment: instance-centric tokens from a frozen open-vocabulary segmenter drive cross-frame attention (InCAM), channel-wise semantic groups index a texture memory (CaTeGory), and the diffusion process combines patch-level blurring with ResShift residual shifting in DWT sub-band space.
Key results
- On REDS4, SeeClear* (extended version) reaches LPIPS 0.1548, a 10.8% relative improvement over the runner-up SATeCo (0.1735), with PSNR 31.32 dB.
- On REDS4 the base SeeClear (5-frame clips) gets PSNR 28.92 and LPIPS 0.1843, trailing regression methods like BasicVSR (31.42 dB) in pixel fidelity.
- Ablation on REDS4 shows InCAM lifts PSNR from 27.99 to 28.46 dB and CaTeGory adds further gains to 28.92 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SeeClear★ instead of SATeCo | REDS4 VSR · REDS4 | LPIPS: 0.1735 -> 0.1548 | Helped |
| use SeeClear★ instead of IconVSR | REDS4 VSR · REDS4 | PSNR: 31.67 -> 31.32 | Hurt |
| use SeeClear★ instead of SATeCo | Vid4 VSR · Vid4 | LPIPS: 0.2291 -> 0.2054 | Helped |
| add intra-frame semantic condition (ablation) | SeeClear baseline with DWT · REDS4 | LPIPS: 0.2120 -> 0.2088 | Helped |
| add intra-frame semantic condition (ablation) | SeeClear baseline with DWT · REDS4 | PSNR: 28.05 -> 28.08 | Helped |
| add MFSA to the semantic-conditioned model (ablation) | SeeClear · REDS4 | PSNR: 28.08 -> 27.99 | Hurt |
| add MFSA to the semantic-conditioned model (ablation) | SeeClear · REDS4 | LPIPS: 0.2088 -> 0.2053 | Helped |
| add InCAM to the semantic-conditioned MFSA model (ablation) | SeeClear · REDS4 | PSNR: 27.99 -> 28.46 | Helped |
| add InCAM to the semantic-conditioned MFSA model (ablation) | SeeClear · REDS4 | LPIPS: 0.2053 -> 0.1917 | Helped |
| add CaTeGory to the semantic-conditioned baseline (ablation) | SeeClear · REDS4 | LPIPS: 0.2088 -> 0.2149 | Hurt |
| add CaTeGory to the semantic-conditioned baseline (ablation) | SeeClear · REDS4 | PSNR: 28.08 -> 28.21 | Helped |
| add DWT to the full semantic model (ablation) | SeeClear · REDS4 | PSNR: 28.74 -> 28.92 | Helped |
Surprising findings
- The base SeeClear is worse than regression-based VSR methods on PSNR/SSIM (e.g., 28.92 vs 31.67 dB for IconVSR on REDS4), trading fidelity for perceptual quality.
- Multi-frame self-attention alone slightly reduces PSNR (28.08 to 27.99 dB) even though it improves LPIPS.
Limitations
- Trained only on video SR datasets, so it does not benefit from large-scale image pretraining and falls short on tiny objects or intricate structures.
- Evaluated only on synthetic bicubic degradation; real-world VSR is left unexplored.
- Wavelet transform replaces the autoencoder without end-to-end learned transforms, which the authors note may limit spatio-temporal modeling.
Experimental setup
Datasets: REDS Vimeo-90K
Benchmarks: REDS4 Vid4
Metrics: PSNR SSIM LPIPS DISTS NIQE CLIP-IQA
Baselines: BasicVSR VRT IconVSR EDVR-M StableSR ResShift SATeCo TOFlow
Builds on: ResShift blurring diffusion OpenSeeD SATeCo
Closest papers by meaning
- Semantic Lens: Instance-Centric Semantic Alignment for Video Super-resolution (AAAI 2024 · similarity 0.90)
- Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution (CVPR 2024 · similarity 0.89)
- One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution (NeurIPS 2025 · similarity 0.89)
- PatchVSR: Breaking Video Diffusion Resolution Limits with Patch-wise Video Super-Resolution (CVPR 2025 · similarity 0.88)
- SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (CVPR 2024 · similarity 0.88)
- VideoGigaGAN: Towards Detail-rich Video Super-Resolution (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.