Research map / Detection and segmentation / CNN features
MCA: Moment Channel Attention Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Moment Channel Attention (MCA), a lightweight channel attention block that pools multiple statistical moments (mean, variance, skewness) via Extensive Moment Aggregation and fuses them with a channel-wise 1D convolution (Cross Moment Convolution).
Problem
Channel attention methods typically squeeze feature maps with global average pooling only (first-order statistics), discarding higher-order distribution information that could improve representation.
Approach
Treats channel activations as samples from a distribution and aggregates first and higher-order central moments (mean, variance, skewness) as the squeeze step, then fuses them with a 1D convolution across channels (CMC) that captures both cross-moment and local cross-channel interaction at negligible parameter cost.
Key results
- On ImageNet with ResNet-50, MCA-S reaches 76.61% Top-1 vs 74.97% baseline and 76.44% for SRM, adding only about 6.06K parameters.
- On COCO with Faster R-CNN ResNet-50, MCA-S achieves 38.3 AP vs 34.9 baseline and 37.9 for GCT.
- On COCO with Mask R-CNN ResNet-50, MCA-E and MCA-S reach 39.0 AP vs 37.5 baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace M1 alone with M2 alone (ablation) | MCA moments aggregation · COCO val2017, Faster R-CNN, ResNet-50 | AP: 36.2 -> 37.6 | Helped |
| Replace M1 alone with M3 alone (ablation) | MCA moments aggregation · COCO val2017, Faster R-CNN, ResNet-50 | AP: 36.2 -> 38.1 | Helped |
| Add M2 to M1 (ablation) | MCA moments aggregation · COCO val2017, Faster R-CNN, ResNet-50 | AP: 36.2 -> 38.0 | Helped |
| Add M3 to M1 (ablation) | MCA moments aggregation · COCO val2017, Faster R-CNN, ResNet-50 | AP: 36.2 -> 38.3 | Helped |
| Add M3 to M2 (ablation) | MCA moments aggregation · COCO val2017, Faster R-CNN, ResNet-50 | AP: 37.6 -> 38.2 | Helped |
| Add M2 and M3 to M1 (ablation) | MCA moments aggregation · COCO val2017, Faster R-CNN, ResNet-50 | AP: 36.2 -> 38.4 | Helped |
| Replace CFC with CMC for moment fusion (ablation) | MCA with ResNet-50 and M1+M2 · COCO val2017, Faster R-CNN | AP: 36.4 -> 38.0 | Helped |
| Use M1+M2+M3 instead of M1+M3 (ablation) | MCA with ResNet-50 · ImageNet | Top-1: 76.61 -> 76.21 | Hurt |
| Use MCA-S instead of GCT | ResNet-50 Faster R-CNN · COCO val2017 | AP: 37.9 -> 38.3 | Helped |
| Use MCA-E instead of GCT | ResNet-101 Faster R-CNN · COCO val2017 | AP: 40.1 -> 40.3 | Helped |
| Use MCA-S instead of GCT | ResNet-50 Mask R-CNN · COCO val2017 | AP: 38.5 -> 39.0 | Helped |
| Use MCA-S instead of GCT | ResNet-101 RetinaNet · COCO val2017 | AP: 38.3 -> 38.8 | Helped |
Surprising findings
- The triple-moment variant M1+M2+M3 performs worse than MCA-E/MCA-S on ImageNet classification (76.21% vs 76.61% Top-1), which the authors attribute to differing value ranges of the moments.
- In the RetinaNet ResNet-50 results, SENet (34.7 AP) falls below the baseline (35.0 AP).
Limitations
- Higher-order moments beyond the third (e.g., kurtosis) were not explored due to computational concerns.
- Triple-moment fusion showed limited or inconsistent gains, and the authors note mixed value ranges across moments may hurt combination.
- Gains on some detectors and backbones are small (e.g., 0.2 AP over GCT on ResNet-101 Faster R-CNN) and the paper reports no variance across seeds.
Experimental setup
Datasets: ImageNet COCO val2017 COCO train
Benchmarks: ImageNet COCO val2017
Metrics: Top-1 accuracy Top-5 accuracy AP AP0.5 AP0.75 APS APM APL Params GFLOPs
Baselines: SENet SRM ECANet GCT
Builds on: Squeeze-and-Excitation Networks ECA-Net SRM GCT Faster R-CNN
Closest papers by meaning
- PartialNet: Compute Less, Perform Better (AAAI 2026 · similarity 0.87)
- Attention Clusters: Purely Attention Based Local Feature Integration for Video Classification (CVPR 2018 · similarity 0.85)
- ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks (CVPR 2020 · similarity 0.85)
- Explicitly Modeled Attention Maps for Image Classification (AAAI 2021 · similarity 0.85)
- Coordinate Attention for Efficient Mobile Network Design (CVPR 2021 · similarity 0.85)
- Degrees of Freedom for Linear Attention: Distilling Softmax Attention with Optimal Feature Efficiency (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.