Research map / Vision-language and multimodal / Cross-modal learning

Multi-Modal Synergistic Implicit Image Enhancement for Efficient Optical Flow Estimation

CVPR 2025 · Vision-language and multimodal › Cross-modal learning · approach: Multi-modality image fusion · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a high-low frequency feature enhancement network for RAFT-based optical flow under low light, implicitly supervised during training by features from a pretrained 2D-3D RGB-depth fusion network.

Problem

Optical flow estimation degrades under adverse lighting (low texture, noise), and paired high-quality images from the same view are hard to obtain for image enhancement.

Approach

An enhancement network (high/low frequency decomposition with dense convs, channel self-attention, wavelet fusion) is trained end-to-end with optical flow loss plus a prior feature loss that aligns its encoder features with those of a pretrained RGB-point cloud 2D-3D fusion network, so depth-derived knowledge is distilled implicitly into RGB-only inference.

Key results

• On the noised FlyingThings3D val set, EPE improves from 4.18 (GMFlow) to 3.85 and ACC1px rises to 76.66%.
• On VBOF, EPE of 19.99 overall versus 20.11 for GMFlow; best on Sony and Fuji subsets.
• Ablation on RAFT: prior feature loss alone reduces EPE 4.41→4.06; enhancement network alone to 4.23; combined 3.85.

Measured effects reported in the paper

Surprising findings

• Prior feature loss from depth-based fusion contributes more to the gain than the enhancement network itself in the ablation.
• On VBOF, the gain over GMFlow is small (about 1%) on several subsets, and the model is second-best on Canon and Nikon.

Limitations

• Gains on real low-light data (VBOF) are modest and uneven across camera subsets.
• Synthetic noise generalization depends on how noise is synthesized, as the authors note.
• Depends on point clouds derived from depth via unprojection, which is not available for all data.

Experimental setup

Datasets: FlyingThings3D FlyingThings3D noised subset VBOF

Benchmarks: FlyingThings3D Sintel VBOF

Metrics: EPE ACC1px parameters (M) memory (GB)

Baselines: RAFT GMA KPAFlow GMFlow CEDFlow

Builds on: RAFT FlowNet GMA CEDFlow

Closest papers by meaning

• Promoting Single-Modal Optical Flow Network for Diverse Cross-Modal Flow Estimation (AAAI 2022 · similarity 0.87)
• Injecting Frame-Event Complementary Fusion into Diffusion for Optical Flow in Challenging Scenes (NeurIPS 2025 · similarity 0.85)
• FlowAnyTime: Efficient Fine-tuning with Intra-Inter Frame Distillation for All-Weather Optical Flow Estimation (AAAI 2026 · similarity 0.85)
• Learning Optical Flow From Still Images (CVPR 2021 · similarity 0.85)
• DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework (CVPR 2025 · similarity 0.85)
• StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences (NeurIPS 2024 · similarity 0.84)