Research map / Detection and segmentation / CNN features
Complex Wavelet Mutual Information Loss: A Multi-Scale Loss Function for Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CWMI loss, which sums mutual information (Gaussian closed-form approximation) between ground-truth and predicted complex steerable pyramid subbands, combined with cross-entropy, for semantic segmentation.
Problem
Pixel-wise losses ignore structural and multiscale information, while region- and boundary-based losses are costly or limited to small regions, leaving class and instance imbalance (small objects, thin boundaries) poorly addressed in semantic segmentation.
Approach
Decompose the prediction and label with a complex steerable pyramid (multi-scale, multi-orientation, with phase), then compute a closed-form Gaussian mutual information per subband using real-valued matrix operations and sum across levels, combined with cross-entropy via a weight lambda.
Key results
- On SNEMI3D with U-Net, CWMI reaches mIoU 0.778 and mDice 0.869, versus 0.773-0.774 for L1/L2 variants and 0.764 for RMI in the ablations/main table.
- On DRIVE with Attention U-Net, CWMI achieves mIoU 0.795 vs 0.785 for RMI (second best in the table).
- CWMI adds about 0.23 s per epoch over CE on U-Net SNEMI3D (2.27 s vs 2.04 s), cheaper than Skea-topo (2.72 s) and clDice (2.37 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the baseline loss with CWMI | U-Net · SNEMI3D | mIoU: .766_{\pm.004} -> .778_{\pm.004} | Helped |
| replace the baseline loss with CWMI | U-Net · GlaS | mIoU: .839_{\pm.011} -> .843_{\pm.016} | Helped |
| replace the baseline loss with CWMI | U-Net · DRIVE | mIoU: .787_{\pm.012} -> .798_{\pm.012} | Helped |
| replace the baseline loss with CWMI | U-Net · MASS ROAD | mIoU: .766_{\pm.008} -> .767_{\pm.010} | No clear effect |
| replace the baseline loss with CWMI | Attention U-Net · DRIVE | mIoU: .785_{\pm.009} -> .795_{\pm.015} | Helped |
| use mutual information rather than L1, L2, or SSIM in the wavelet domain (ablation) | U-Net · SNEMI3D | mIoU: .773_{\pm.002}, .774_{\pm.003}, .468_{\pm.032} -> .779_{\pm.004} | Helped |
| combine magnitude and phase instead of using phase only (ablation) | CWMI · SNEMI3D | mIoU: .694_{\pm.008} -> .778_{\pm.004} | Helped |
| combine magnitude and phase instead of using magnitude only (ablation) | CWMI · SNEMI3D | mIoU: .775_{\pm.004} -> .778_{\pm.004} | Helped |
| set λ=0.5 rather than λ=0.1 (ablation) | CWMI · SNEMI3D | mIoU: .778_{\pm.004} -> .778_{\pm.006} | No clear effect |
| set λ=0.9 rather than λ=0.1 (ablation) | CWMI · SNEMI3D | mIoU: .778_{\pm.004} -> .777_{\pm.003} | No clear effect |
| use K=4 rather than K=2 (ablation) | CWMI · SNEMI3D | mIoU: .777_{\pm.006} -> .779_{\pm.004} | Helped |
| use K=4 rather than K=8 (ablation) | CWMI · SNEMI3D | mIoU: .776_{\pm.003} -> .779_{\pm.004} | Helped |
Surprising findings
- Phase-only CWMI performs substantially worse than magnitude or real-only variants (mIoU 0.694 vs 0.775-0.776 on SNEMI3D).
- Several baseline losses (ABW, Skea-topo, clDice, Sensitive) failed to converge on some datasets, reported as hyphens.
Limitations
- Evaluated only on 2D binary-style segmentation; multi-class and 3D extension is untested.
- Mutual information is symmetric and cannot distinguish correct from inverted label assignments, so cross-entropy is required.
- Small datasets and limited hardware (single A100 via Google Colab); statistical significance varies across metrics.
Experimental setup
Datasets: SNEMI3D GlaS DRIVE MASS ROAD
Benchmarks: SNEMI3D GlaS DRIVE MASS ROAD
Metrics: mIoU mDice VI ARI HD
Baselines: Cross entropy BCE Focal loss Dice loss Tversky loss Jaccard loss Sensitive loss WCE ABW loss Skea-topo loss RMI loss clDice loss
Builds on: Regional Mutual Information (RMI) loss steerable pyramid complex steerable pyramid U-Net Attention U-Net
Closest papers by meaning
- Region Mutual Information Loss for Semantic Segmentation (NeurIPS 2019 · similarity 0.88)
- Consistent-Separable Feature Representation for Semantic Segmentation (AAAI 2021 · similarity 0.87)
- Integrating Low-Level Visual Cues for Enhanced Unsupervised Semantic Segmentation (AAAI 2025 · similarity 0.85)
- Separate and Conquer: Decoupling Co-occurrence via Decomposition and Representation for Weakly Supervised Semantic Segmentation (CVPR 2024 · similarity 0.84)
- Hierarchy-Agnostic Unsupervised Segmentation: Parsing Semantic Image Structure (NeurIPS 2024 · similarity 0.84)
- Structural Pruning via Spatial-aware Information Redundancy for Semantic Segmentation (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.