Research map / Detection and segmentation / Weakly supervised detection and segmentation
Weakly Supervised Virus Capsid Detection with Image-Level Annotations in Electron Microscopy Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Weakly supervised virus capsid detection in EM images that uses only image-level presence/absence labels, optimizing Gaussian-masked classifier scores with a shrinking receptive field to produce pseudo-labels for training a Faster-RCNN.
Problem
Object detection of virus particles in electron microscopy requires expensive expert bounding-box annotations; weakly supervised methods relying on CAMs or ROI proposals perform poorly on small, noisy EM datasets with many instances per image.
Approach
Treat virus localization as gradient-based optimization of a Gaussian mask's center on a frozen image-level classifier, initialized from GradCAM hotspots and annealing the Gaussian std from large to small (score-based-generative-model style), iteratively removing detected capsids; the resulting pseudo-boxes can train a standard detector.
Key results
- Ours (OD) reaches 91.20 mAP50 on Herpes vs 89.18 for fully supervised BB, and 58.28 / 74.32 / 78.33 / 78.34 on Adeno / Noro / Papilloma / Rota, outperforming weak baselines such as CutLER and Reattention.
- Under a fixed annotation-time budget, Ours (Opt) outperforms location and bounding-box labels when annotation time is below 25% of the full budget on Herpes.
- A user study with six experts found binary image-level labels were faster to annotate (11 h vs 19 h for boxes on Herpes) and less error-prone than location or bounding-box labels.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train a detector on image-level-derived pseudo boxes (Ours (OD)) instead of fully supervised bounding-box labels | Faster R-CNN virus detection · Herpes virus | mAP50: 89.18 { $\pm 0.95$} -> 91.20 { $\pm 0.24$} | Helped |
| Train a detector on image-level-derived pseudo boxes (Ours (OD)) rather than use the strongest listed baseline | Virus detection · Adeno virus | mAP50: 68.49 { $\pm 5.44$} -> 58.28 { $\pm 5.91$} | Hurt |
| Train a detector on image-level-derived pseudo boxes (Ours (OD)) rather than use the strongest listed baseline | Virus detection · Noro virus | mAP50: 55.09 { $\pm 8.92$} -> 74.32 { $\pm 1.18$} | Helped |
| Train a detector on image-level-derived pseudo boxes (Ours (OD)) rather than use the strongest listed baseline | Virus detection · Papilloma virus | mAP50: 73.23 { $\pm 7.02$} -> 78.33 { $\pm 2.40$} | Helped |
| Train a detector on image-level-derived pseudo boxes (Ours (OD)) rather than use the strongest listed baseline | Virus detection · Rota virus | mAP50: 75.5 { $\pm 1.44$} -> 78.34 { $\pm 2.15$} | Helped |
| Use GradCAM initialization instead of Selective Search (ablation) | Ours (Opt) · Herpes virus validation set | mAP50: 66.77 { $\pm 7.28$} -> 82.40 { $\pm 0.25$} | Helped |
| Use GradCAM initialization instead of random initialization (ablation) | Ours (Opt) · Herpes virus validation set | mAP50: 60.24 { $\pm 2.29$} -> 82.40 { $\pm 0.25$} | Helped |
| Use ResNet-101 instead of ViT-B/16 (ablation) | Ours (Opt) · Herpes virus | mAP50: 48.66 { $\pm 07.44$} -> 86.98 { $\pm 1.92$} | Helped |
| Use logits instead of classifier score in the optimization loss (ablation) | Ours (Opt) · Herpes virus validation set | mAP50: 75.75 { $\pm 2.60$} -> 76.72 { $\pm 1.62$} | Helped |
| Use mean masking instead of zero masking (ablation) | Ours (Opt) · Herpes virus validation set | mAP50: 72.01 { $\pm 2.32$} -> 76.72 { $\pm 1.62$} | Helped |
| Use mean masking instead of inpainting (ablation) | Ours (Opt) · Herpes virus validation set | mAP50: 75.70 { $\pm 1.70$} -> 76.72 { $\pm 1.62$} | Helped |
| Score a detection while masking other detected viruses rather than masking the background (ablation) | Ours (Opt) · Herpes virus validation set | mAP50: 76.72 { $\pm 1.62$} -> 80.35 { $\pm 0.80$} | Helped |
Surprising findings
- With infinite annotation time and small datasets, fully supervised Loc labels beat the proposed Ours (OD) on Adeno, Noro, Papilloma and Rota, so the advantage is limited to tight annotation budgets.
- ViT backbones performed much worse than ResNet-101 for the proposed optimization (e.g., 48.66 vs 86.98 mAP50 on Herpes), attributed to small datasets.
Limitations
- Performance depends heavily on classifier quality; pseudo-labels produce false positives that hurt detector training on small datasets.
- Requires known virus size in nm and assumes a fixed object size; a 30% size error drops mAP50 from 86.98 to 67.31.
- Does not handle objects of varying size, as noted by the authors.
Experimental setup
Datasets: Herpes virus TEM dataset (359 images) Adeno virus negative stain TEM (67 images) Noro virus negative stain TEM (54 images) Papilloma virus negative stain TEM (31 images) Rota virus negative stain TEM (36 images)
Benchmarks: Herpes virus Adeno virus Noro virus Papilloma virus Rota virus
Metrics: mAP50 F1 score (user study) annotation time forward passes
Baselines: Faster R-CNN fully supervised (BB) Loc minimal-label detector SAM CutLER GradCAM LayerCAM TS-CAM Reattention DoG
Builds on: GradCAM Faster R-CNN CutLER Segment Anything Model Reattention
Closest papers by meaning
- DLWL: Improving Detection for Lowshot Classes With Weakly Labelled Data (CVPR 2020 · similarity 0.84)
- Partial Weakly-Supervised Oriented Object Detection (CVPR 2026 · similarity 0.84)
- Parsing Images of Overlapping Organisms With Deep Singling-Out Networks (CVPR 2017 · similarity 0.83)
- Relational Matching for Weakly Semi-Supervised Oriented Object Detection (CVPR 2024 · similarity 0.83)
- UWSOD: Toward Fully-Supervised-Level Capacity Weakly Supervised Object Detection (NeurIPS 2020 · similarity 0.83)
- Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.