Research map / Image generation / Image restoration
LDP: Language-driven Dual-Pixel Image Defocus Deblurring Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uses CLIP (ViT-B/32, frozen) with symmetry-based text prompts on flipped dual-pixel pairs to estimate blur maps unsupervisedly, then feeds them to a NAFNet-based deblurring network with BPA attention blocks and blur-weighting/blur-aware losses.
Problem
Single-image-pair defocus deblurring of dual-pixel (DP) pairs with disparity-dependent blur, where blur map estimation previously required extra supervision or pre-calibrated kernels.
Approach
Zero-shot blur map estimation by feeding a monocular-trained CLIP an image formed by concatenating the left view with a flipped right view and querying "horizontally symmetric" text prompts (a sharp pixel is horizontally symmetric), then using the blur map as a deblurring kernel prior via a BPA block and as a loss weight and regularizer.
Key results
- Ours (Large) reaches 27.24 dB PSNR on DPD-blur, above K3DN (27.06) and Restormer (26.66).
- Ours (Large) reaches 38.62 dB on DDD-syn and 33.44 dB on RDPD, the best reported PSNR on both.
- Inference time with CLIP is 0.042 s (small) and 0.054 s (large) on an RTX 4090, versus 0.198 s for K3DN and 0.255 s for Restormer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use blur-map kernel priors in the BPA block instead of concatenating the blur map and using standard self-attention (ablation) | NAFNet-based deblurring network · DPD-blur | PSNR: 26.80 -> 26.91 | Helped |
| add blur-weighting loss (ablation) | BPA block deblurring network with Charbonnier loss · DPD-blur | PSNR: 26.71 -> 26.76 | Helped |
| add blur-aware loss (ablation) | BPA block deblurring network with Charbonnier loss · DPD-blur | PSNR: 26.71 -> 26.81 | Helped |
| add both blur-weighting and blur-aware losses (ablation) | BPA block deblurring network · DPD-blur | PSNR: 26.71 -> 26.91 | Helped |
| ensemble all blur-aware and DP-aware prompts instead of using a single prompt (ablation) | LDP blur-map estimation · DPD-blur | PSNR: 26.85 -> 26.91 | Helped |
| scale from the small model to the large model | LDP · DPD-blur average | PSNR (dB): 26.91 -> 27.24 | Helped |
| scale from the small model to the large model | LDP · DDD-syn | PSNR: 37.27 -> 38.62 | Helped |
| scale from the small model to the large model | LDP · RDPD | PSNR: 32.98 -> 33.44 | Helped |
Surprising findings
- The DP-aware symmetry prompt, which uses no blur wording, is the best single prompt family, outperforming the blur-aware prompts that the method's framing would suggest.
Limitations
- Texture-less regions (walls, ceilings, floors) produce weak CLIP responses and degrade the estimated blur map.
- Blur map accuracy on DPD-disp is 89.8% versus 88.3% for DPD-disp and 70.2% for BAMBNet, evaluated against a thresholded pseudo ground truth.
Experimental setup
Datasets: DPD-blur DPD-disp DDD-syn RDPD
Benchmarks: DPD-blur DDD-syn RDPD DPD-disp
Metrics: PSNR SSIM MAE RMSE rel LPIPS
Baselines: EBDB DMENet DPDNet RDPD DDDNet IFAN BAMBNet DeepRFT Restormer K3DN
Builds on: CLIP NAFNet DPDNet
Closest papers by meaning
- Quad-Pixel Image Defocus Deblurring: A New Benchmark and Model (CVPR 2025 · similarity 0.88)
- Blur2Blur: Blur Conversion for Unsupervised Image Deblurring on Unknown Domains (CVPR 2024 · similarity 0.87)
- Residual Diffusion Deblurring Model for Single Image Defocus Deblurring (AAAI 2025 · similarity 0.86)
- Real-World Efficient Blind Motion Deblurring via Blur Pixel Discretization (CVPR 2024 · similarity 0.86)
- BlurDM: A Blur Diffusion Model for Image Deblurring (NeurIPS 2025 · similarity 0.85)
- Multi-Focus Image Fusion via Explicit Defocus Blur Modelling (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.