Research map / Detection and segmentation / Weakly supervised detection and segmentation
ToNNO: Tomographic Reconstruction of a Neural Network's Output for Weakly Supervised Segmentation of 3D Medical Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ToNNO, which trains a 2D slice classifier on volume-level labels and reconstructs a 3D segmentation heatmap via inverse Radon transform, plus Averaged and Tomographic CAM variants, for weakly supervised 3D medical image segmentation.
Problem
Weakly supervised segmentation of 3D medical images using only image-level (volume-level) presence/absence labels for regions such as tumours or lesions, where existing methods mostly rely on 2D CAM and yield low-resolution heatmaps.
Approach
Train a 2D encoder on randomly oriented slices of 3D volumes with volume-level labels, then replace the sum in the Radon transform with the network output and apply the inverse 3D-2D Radon transform (filtered backprojection) to reconstruct a 3D heatmap; also average CAM maps across many rotations, with a second-derivative filtering step (Tomographic CAM) to keep deep-layer maps sharp.
Key results
- Tomographic LayerCAM achieves the best F1-score, with an average improvement of 0.14 over LayerCAM across the four datasets.
- On Multiple Sclerosis, ToNNO improves Dice over LayerCAM from 0.41 to 0.53 (+0.12).
- On Duke, ToNNO raises balanced accuracy from 0.58 to 0.79 and F1 from 0.24 to 0.51 relative to LayerCAM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ToNNO instead of LayerCAM | ResNet-10-T, frozen batch norm · Multiple Sclerosis | F1-score: 0.73 -> 0.82 | Helped |
| Use ToNNO instead of LayerCAM | ResNet-10-T, frozen batch norm · AutoPET | F1-score: 0.35 -> 0.39 | Helped |
| Use ToNNO instead of LayerCAM | ResNet-10-T, frozen batch norm · MosMed | F1-score: 0.39 -> 0.50 | Helped |
| Use ToNNO instead of LayerCAM | ResNet-10-T, frozen batch norm · Duke | F1-score: 0.24 -> 0.51 | Helped |
| Apply tomographic reconstruction to LayerCAM heatmaps instead of standard LayerCAM | ResNet-10-T, frozen batch norm · AutoPET | F1-score: 0.35 -> 0.49 | Helped |
| Apply tomographic reconstruction to LayerCAM heatmaps instead of standard LayerCAM | ResNet-10-T, frozen batch norm · MosMed | F1-score: 0.39 -> 0.55 | Helped |
| Apply tomographic reconstruction to LayerCAM heatmaps instead of standard LayerCAM | ResNet-10-T, frozen batch norm · Multiple Sclerosis | F1-score: 0.73 -> 0.84 | Helped |
| Apply tomographic reconstruction to LayerCAM heatmaps instead of standard LayerCAM | ResNet-10-T, frozen batch norm · Duke | F1-score: 0.24 -> 0.51 | Helped |
| Add pretrained weights (ablation) | ResNet-10-T · All four datasets | F1-score: 0.28 -> 0.55 | Helped |
| Freeze batch-normalization layers rather than use vanilla batch normalization (ablation) | ResNet-10-T · All four datasets | F1-score: 0.48 -> 0.55 | Helped |
| Replace batch normalization with group normalization (ablation) | ResNet-10-T · All four datasets | F1-score: 0.48 -> 0.54 | Helped |
| Use ResNet-50 with group normalization instead of ResNet-10-T with frozen batch normalization (ablation) | ToNNO classifier · All four datasets | F1-score: 0.55 -> 0.56 | No clear effect |
Surprising findings
- Batch normalization harms the classifier for tomographic reconstruction; frozen BN or group norm works better, and a ResNet-10-T with random initialization performs much worse.
Limitations
- Reconstruction is slower than slice-wise CAM methods, requiring many inferences per volume.
- Tomographic and Averaged CAM are not perfect generalizations of ToNNO; deep-layer CAM maps remain blurry in intermediate cases.
- Tomographic reconstructions are too sharp for large lesions, leading to undersegmentation on MosMed.
Experimental setup
Datasets: Multiple Sclerosis brain MRI (private) AutoPET-II MosMedData COVID-19 Duke breast cancer MRI
Benchmarks: Multiple Sclerosis AutoPET-II MosMedData COVID-19 Duke breast cancer MRI
Metrics: precision recall F1-score Dice MaxIoU balanced accuracy
Baselines: GradCAM LayerCAM Averaged LayerCAM
Builds on: GradCAM LayerCAM ResNet
Closest papers by meaning
- Integrating Deep Metric Learning with Coreset for Active Learning in 3D Segmentation (NeurIPS 2024 · similarity 0.86)
- SGTC: Semantic-Guided Triplet Co-training for Sparsely Annotated Semi-Supervised Medical Image Segmentation (AAAI 2025 · similarity 0.85)
- Upping the Game: How 2D U-Net Skip Connections Flip 3D Segmentation (NeurIPS 2024 · similarity 0.84)
- EffiDec3D: An Optimized Decoder for High-Performance and Efficient 3D Medical Image Segmentation (CVPR 2025 · similarity 0.84)
- Revisiting MAE Pre-training for 3D Medical Image Segmentation (CVPR 2025 · similarity 0.84)
- UniPro: Unified Multi-Mode Medical Image Segmentation from 2D Images to 3D Volumes via Propagation (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.