Research map / Vision-language and multimodal / Cross-modal learning
Learning Based Multi-Modality Image and Video Compression
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use cross-modality reference information with channel-wise and spatial-wise feature alignment | Image compression · FLIR visible | BDBR vs BPG: -22.342 -> -30.226 | Helped |
| Use cross-modality reference information with channel-wise and spatial-wise feature alignment | Image compression · FLIR infrared | BDBR vs BPG: -14.960 -> -21.621 | Helped |
| Use cross-modality reference information with channel-wise and spatial-wise feature alignment | Image compression · KAIST visible | BDBR vs BPG: -3.624 -> -18.639 | Helped |
| Use cross-modality reference information with channel-wise and spatial-wise feature alignment | Image compression · KAIST infrared | BDBR vs BPG: -8.751 -> -21.289 | Helped |
| Add channel-wise feature alignment (ablation) | Multi-modality image compression · KAIST visible | PSNR: more than 0.4dB | Helped |
| Add spatial-wise feature alignment (ablation) | Multi-modality image compression · KAIST visible | PSNR: nearly 0.3dB | Helped |
| Combine channel-wise and spatial-wise feature alignment (ablation) | Multi-modality image compression · KAIST visible | PSNR: more than 0.7dB | Helped |
| Encode the residual between the aligned feature and visible feature instead of concatenating them (ablation) | Channel-wise alignment model · KAIST visible | PSNR: 0.14dB | Hurt |
| Simply concatenate infrared and visible images as input (ablation) | Multi-modality image compression · KAIST visible | PSNR | No clear effect |
| Use element-wise affine coefficients instead of channel-wise coefficients (ablation) | Multi-modality image compression · KAIST visible | PSNR: more than 6dB | Hurt |
| Jointly optimize infrared and visible image compression end to end (ablation) | Multi-modality image compression · Image compression | PSNR: 0.02db | No clear effect |
| Use infrared video frames as cross-modality information for visible video compression | Multi-modality video compression · KAIST video | PSNR: about 0.7dB and 3dB | Helped |
Closest papers by meaning
- End-to-End RGB-D Image Compression via Exploiting Channel-Modality Redundancy (AAAI 2024 · similarity 0.87)
- Text-Guided Channel Perturbation and Pre-Trained Knowledge Integration for Unified Multi-Modality Image Fusion (AAAI 2026 · similarity 0.86)
- Modality-Agnostic Variational Compression of Implicit Neural Representations (ICML 2023 · similarity 0.85)
- Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion (ICLR 2025 · similarity 0.84)
- Extending Multi-modal Contrastive Representations (NeurIPS 2024 · similarity 0.82)
- CrossMAE: Cross-Modality Masked Autoencoders for Region-Aware Audio-Visual Pre-Training (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.