Research map / Detection and segmentation / Weakly supervised detection and segmentation
MoRe: Class Patch Attention Needs Regularization for Weakly Supervised Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MoRe, a single-stage ViT-based WSSS method that regularizes class-patch attention via a directed-graph category representation and localization-informed contrastive/uncertainty losses to remove LAM artifacts.
Problem
Localization attention maps (LAM) from class-patch attention in ViTs suffer from artifacts, where low-information patches are falsely activated by class tokens, under image-level weak supervision for semantic segmentation.
Approach
Views class-patch attention as a learnable directed graph (heads, tails, edges) with top-K neighbor selection, and uses CAM-derived confident and uncertain relation maps to explicitly supervise class-patch attention with contrastive-style losses, trained end-to-end.
Key results
- MoRe achieves 76.4% mIoU on VOC val and 75.0% on VOC test, and 47.4% on COCO val, outperforming single-stage ToCo (71.1 val) and DuPL (73.3 val) and some multi-stage methods.
- MoRe-LAM pseudo labels reach 77.0% mIoU without DenseCRF and 80.0% with DenseCRF on VOC train, versus 76.9/79.7 for MoRe-CAM.
- Ablation on VOC val: removing L_cre drops mIoU from 76.4 to 65.0; removing GCR drops it to 72.1; the ViT baseline reaches 41.6.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MoRe class-patch attention regularization | ViT-B WSSS · PASCAL VOC val | mIoU: 73.3 -> 76.4 | Helped |
| add MoRe class-patch attention regularization | ViT-B WSSS · PASCAL VOC test | mIoU: 72.8 -> 75.0 | Helped |
| add MoRe class-patch attention regularization | ViT-B WSSS · MS COCO val | mIoU: 46.3 -> 47.4 | Helped |
| add L_cre supervision (ablation) | MoRe on ViT-B · PASCAL VOC val | mIoU: 65.0 -> 76.4 | Helped |
| add L_cre supervision (ablation) | MoRe on ViT-B · PASCAL VOC val | Precision: 75.3 -> 84.5 | Helped |
| add L_ure supervision (ablation) | MoRe on ViT-B · PASCAL VOC val | Recall: 85.7 -> 88.0 | Helped |
| add GCR (ablation) | MoRe on ViT-B · PASCAL VOC val | mIoU: 72.1 -> 76.4 | Helped |
| add L_ure supervision (ablation) | MoRe on ViT-B · PASCAL VOC val | mIoU: 74.4 -> 76.4 | Helped |
Surprising findings
- Applying the PTC over-smoothness loss to the ViT baseline does not remove LAM artifacts, suggesting artifacts stem from class-patch relations rather than patch over-smoothness.
Limitations
- Relies on CAM from the classification head to derive confident and uncertain relation maps, so quality depends on CAM quality.
- Many hyperparameters (top-K=392, kernel size, threshold phi, loss weights) need tuning; the sensitivity analysis is deferred to the appendix.
Experimental setup
Datasets: PASCAL VOC 2012 MS COCO 2014 ImageNet
Benchmarks: PASCAL VOC 2012 MS COCO 2014
Metrics: mIoU precision recall confusion ratio (FP/TP)
Baselines: MCTformer+ CLIP-ES CPAL ToCo DuPL AFA ViT-PCM CLIMS L2G RIB
Builds on: MCTformer TS-CAM ToCo DeepLab-LargeFOV
Closest papers by meaning
- Multi-Class Token Transformer for Weakly Supervised Semantic Segmentation (CVPR 2022 · similarity 0.91)
- Class Tokens Infusion for Weakly Supervised Semantic Segmentation (CVPR 2024 · similarity 0.88)
- W2P: Switching from Weak Supervision to Partial Supervision for Semantic Segmentation (AAAI 2024 · similarity 0.86)
- Weakly Supervised Semantic Segmentation via Progressive Confidence Region Expansion (CVPR 2025 · similarity 0.86)
- Attention Guided CAM: Visual Explanations of Vision Transformer Guided by Self-Attention (AAAI 2024 · similarity 0.86)
- Finer-CAM: Spotting the Difference Reveals Finer Details for Visual Explanation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.