Research map / Architectures and efficiency / Neural architecture search

LoMix: Learnable Weighted Multi-Scale Logits Mixing for Medical Image Segmentation

NeurIPS 2025 · Architectures and efficiency › Neural architecture search · approach: AutoML pipeline search · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LoMix, a training-only module that fuses multi-scale decoder logits with four operators (add, mult, concat, attention-weighted fusion) and learns softplus loss weights for all original and mutant maps, improving Synapse, ACDC and other segmentation benchmarks at zero inference cost.

Problem

Standard training of U-shaped segmentation networks either supervises only the final logits or applies deep supervision with equal weights across scales, ignoring complementary information from mixed-scale logit combinations, especially under limited labels.

Approach

Generate mixed-scale "mutant" logits by applying four fusion operators to every subset of decoder logits, then supervise all original and mutant maps with learnable softplus loss weights optimized jointly with network parameters, acting as a training-only NAS-style weighting with no inference overhead.

Key results

• On Synapse 8-organ with PVT-EMCAD-B2, LoMix reaches 85.1% average DICE vs 80.9% last-layer and 82.9% deep supervision in the main table.
• With only 5% of Synapse training data (1 scan), LoMix improves DICE from 37.22% to 46.45% over last-layer supervision.
• On ACDC, PVT-EMCAD-B2 + LoMix attains 92.51% average DICE vs 92.12% for the PVT-EMCAD-B2 baseline.

Measured effects reported in the paper

Surprising findings

• Learned weights concentrate on attention-weighted fusion and multiplicative mixes, while concatenation receives almost negligible weight at convergence.
• Gains over MUTATION (equal-weight additive fusion) are relatively modest on several backbones, e.g. +1.5% DICE for PVT-EMCAD-B2.

Limitations

• Implementation limited to 2D medical segmentation tasks.
• Number of mutant logits grows as 4(2^L - 1 - L), limiting scalability to many decoder stages.
• Reported gains over MUTATION are small on some backbones and some per-organ results are mixed, with variance reported as ±0.9 to ±1.3 in supplementary tables.

Experimental setup

Datasets: Synapse multi-organ ACDC BUSI ISIC2018 Kvasir CVC-ColonDB ETIS-LaribPolypDB

Benchmarks: Synapse 8-organ Synapse 13-organ ACDC BUSI ISIC2018 Kvasir CVC-ColonDB ETIS-LaribPolypDB

Metrics: DICE HD95 mIoU

Baselines: Last Layer supervision Deep Supervision MUTATION UNet AttUNet TransUNet UNeXt PVT-CASCADE-B2 PVT-EMCAD-B2

Builds on: deep supervision UNet++ MERIT MUTATION EMCAD PVT-v2 uncertainty weighting

Closest papers by meaning

• MedSAMix: A Training-Free Model Merging Approach for Medical Image Segmentation (AAAI 2026 · similarity 0.86)
• FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks (ICML 2025 · similarity 0.86)
• Rethinking U-Net: Task-Adaptive Mixture of Skip Connections for Enhanced Medical Image Segmentation (AAAI 2025 · similarity 0.85)
• EMCAD: Efficient Multi-scale Convolutional Attention Decoding for Medical Image Segmentation (CVPR 2024 · similarity 0.85)
• Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decoding (CVPR 2024 · similarity 0.85)
• nnWNet: Rethinking the Use of Transformers in Biomedical Image Segmentation and Calling for a Unified Evaluation Benchmark (CVPR 2025 · similarity 0.84)