Research map / 3D vision / 3D generation
Cross-Dimension Affinity Distillation for 3D EM Neuron Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a lightweight 2D Y-shape network that models 3D neuron affinities from adjacent sections, trained with cross-dimension affinity distillation from a 3D teacher and a feature grafting interaction module, achieving better VOI at about 1/20 the inference latency.
Problem
3D CNN affinity-based neuron segmentation from EM volumes is computationally expensive and limited in input size, making large-scale deployment impractical.
Approach
Replace 3D affinity prediction with a 2D Y-shaped network that outputs embeddings for adjacent sections whose cosine distances give 3D affinities, and transfer inter-section dependencies from a jointly trained 3D teacher via online cross-dimension affinity distillation plus a feature grafting interaction loss.
Key results
- On AC3/4 with Waterz post-processing, the method reaches VOI 0.8835 vs 0.9764 for the second-best APViT, with 17.29 s latency vs 113.83 s for Superhuman.
- On Wafer4 with Waterz, VOI 0.5587 vs 0.5930 for PEA, about 5.7% lower than the second best.
- Model has 0.88M parameters and 7509 GMAC, versus 1.48M and 147520 GMAC for Superhuman.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CAD to the 2D network (ablation) | 2D affinity network without CAD or FGI · AC3/4, Waterz | VOI: 1.0317 -> 0.9259 | Helped |
| Add FGI to the 2D network with CAD (ablation) | 2D affinity network with CAD · AC3/4, Waterz | VOI: 0.9259 -> 0.8835 | Helped |
| Add CAD to the 2D network (ablation) | 2D affinity network without CAD or FGI · AC3/4, LMC | VOI: 1.1023 -> 0.9490 | Helped |
| Add FGI to the 2D network with CAD (ablation) | 2D affinity network with CAD · AC3/4, LMC | VOI: 0.9490 -> 0.9212 | Helped |
| Add CAD to the 3D network (ablation) | 3D affinity network without CAD or FGI · AC3/4, Waterz | VOI: 1.0502 -> 0.9460 | Helped |
| Add FGI to the 3D network with CAD (ablation) | 3D affinity network with CAD · AC3/4, Waterz | VOI: 0.9460 -> 0.8838 | Helped |
| Add CAD to the 3D network (ablation) | 3D affinity network without CAD or FGI · AC3/4, LMC | VOI: 1.1112 -> 1.0614 | Helped |
| Add FGI to the 3D network with CAD (ablation) | 3D affinity network with CAD · AC3/4, LMC | VOI: 1.0614 -> 0.9977 | Helped |
| Set CAD and FGI loss weights to 1.0 (ablation) | Proposed framework · AC3/4 validation set, Waterz | VOI: 0.6572 -> 0.5915 -> 0.6765 | Helped |
| Set CAD and FGI loss weights to 1.0 (ablation) | Proposed framework · AC3/4 validation set, LMC | VOI: 0.7552 -> 0.6705 -> 0.6941 | Helped |
Surprising findings
- The 2D student without CAD and FGI already beats several 3D baselines on some CREMI and AC3/4 VOI metrics.
- CAD also improved the 3D teacher network, not only the 2D student.
Limitations
- Gains on some datasets are small or absent; e.g., Ours w/o KD is worse than baselines on some LMC VOI results.
- Requires training a 3D teacher network jointly, so training cost is higher than the inference cost suggests.
- Hyperparameters tuned on AC3/4 validation and only three datasets evaluated.
Experimental setup
Datasets: CREMI AC3/4 Wafer4
Benchmarks: CREMI-A CREMI-B CREMI-C AC3/4 Wafer4
Metrics: VOI VOI split VOI merge ARAND latency FLOPs parameters
Baselines: Superhuman MALA PEA APViT Waterz LMC
Builds on: Superhuman PEA deep mutual learning MALA
Closest papers by meaning
- Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing (AAAI 2024 · similarity 0.90)
- Efficient Connectivity-Preserving Instance Segmentation with Supervoxel-Based Loss Function (AAAI 2025 · similarity 0.85)
- Electron Microscopy Images as Set of Fragments for Mitochondrial Segmentation (AAAI 2024 · similarity 0.85)
- FISBe: A Real-World Benchmark Dataset for Instance Segmentation of Long-Range Thin Filamentous Structures (CVPR 2024 · similarity 0.84)
- $\epsilon$-Seg: Sparsely Supervised Semantic Segmentation of Microscopy Data (NeurIPS 2025 · similarity 0.84)
- Reproducibility of predictive networks for mouse visual cortex (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.