Research map / Architectures and efficiency / RNNs and normalization
GliaNet: Adaptive Neural Network Structure Learning with Glia-Driven
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GliaNet, a neural network whose structure and connections are adaptively optimized during training via a Glia-Neuron model (oligodendrocyte neuron selection plus astrocyte connection modulation), reducing parameters while improving accuracy on classification, segmentation and detection.
Problem
Fixed M-P neural network structures cause over-fitting and over-parameterization, and prior astrocyte-based and pruning methods remove connections empirically, which can hurt accuracy.
Approach
Introduces a Glia unit (oligodendrocyte selects winning neurons, astrocyte modulates their connections) organized into groups with temporal and inter-unit interactions, implemented with LSTMs, pooling and a ViT-based integration function, and trained with alternating NM/GM optimization plus a contrastive regularizer, replacing hard empirical pruning.
Key results
- On CIFAR10, GliaNet (RNT18) reaches 96.90% accuracy with 6.95M params versus 94.96% for ResNet-18 at 11.17M, and beats MA-Net by 0.09% on RNT18 and 0.17% on RNT50.
- On ImageNet-1k, Ours (RNT50) reaches 79.15% with 8.15M params, versus 78.96% for MA-Net (8.16M) and 75.30% for ResNet-50 (26.11M).
- On COCO with RNT50 backbone, AP improves to 41.5 for instance segmentation (CondInst setup) and 40.5 for Faster RCNN detection, over MA-Net's 40.7 and 39.8 respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GliaNet instead of MA-Net with RNT18 | RNT18 · CIFAR10 | accuracy (%): 96.81 -> 96.90 | Helped |
| use GliaNet instead of MA-Net with RNT50 | RNT50 · CIFAR10 | accuracy (%): 97.10 -> 97.26 | Helped |
| use GliaNet instead of MA-Net with WRN50 | WRN50 · CIFAR10 | accuracy (%): 97.25 -> 97.42 | Helped |
| use GliaNet instead of MA-Net with RNT50 | RNT50 · ImageNet-1k | accuracy (%): 78.96 -> 79.15 | Helped |
| use GliaNet-pretrained backbone instead of MA-Net-pretrained backbone | BoxInst · COCO | AP: 34.7 -> 35.7 | Helped |
| use GliaNet-pretrained backbone instead of MA-Net-pretrained backbone | CondInst · COCO | AP: 40.7 -> 41.5 | Helped |
| use GliaNet-pretrained backbone instead of MA-Net-pretrained backbone | Faster RCNN · COCO | AP: 39.8 -> 40.5 | Helped |
| use GliaNet-pretrained backbone instead of MA-Net-pretrained backbone | Dynamic DETR · COCO | AP: 44.0 -> 45.1 | Helped |
| use GliaNet with ViT-B | ViT-B · ImageNet-1k | accuracy (%): 77.90 -> 79.52 | Helped |
| use GliaNet with Swin-B | Swin-B · ImageNet-1k | accuracy (%): 84.50 -> 85.70 | Helped |
| use GliaNet-pretrained ViT-B instead of ViT-B | ViT-B · CIFAR100 fine-tuning | accuracy (%): 87.13 -> 88.05 | Helped |
| use GliaNet-pretrained ViT-L instead of ViT-L | ViT-L · CIFAR100 fine-tuning | accuracy (%): 86.35 -> 87.20 | Helped |
Surprising findings
- Ablation shows that using multiple astrocytes alone (Case 3, 96.81%) is nearly as strong as the full method, while multiple oligodendrocytes alone gives 96.72%.
- Increasing the number of Glia units monotonically improves CIFAR10 accuracy (96.68% at 1 unit to 96.90% at 8), but training time grows substantially.
Limitations
- Training time grows significantly with the number of Glia units, leading to a trade-off with accuracy.
- Improvements over the strongest prior method (MA-Net) are often small (e.g., 0.09% to 0.17% on CIFAR10).
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet-1k COCO
Benchmarks: CIFAR10 CIFAR100 ImageNet-1k COCO
Metrics: top-1 accuracy AP parameters (M) inference speed (ms) GPU days
Baselines: ResNet Wide ResNet CDS AstroNet MA-Net NAS-LID HOTNAS Boost-DyNN BoxInst CondInst Faster RCNN Dynamic DETR
Builds on: Astrocyte-Neuron model (AstroNet) MA-Net CDS contrastive dynamic structure CondInst BoxInst
Closest papers by meaning
- MA-Net: Rethinking Neural Unit in the Light of Astrocytes (AAAI 2024 · similarity 0.88)
- SPINT: Spatial Permutation-Invariant Neural Transformer for Consistent Intracortical Motor Decoding (NeurIPS 2025 · similarity 0.82)
- Neuroformer: Multimodal and Multitask Generative Pretraining for Brain Data (ICLR 2024 · similarity 0.82)
- BioOSS: A Bio-Inspired Oscillatory State System with Spatio-Temporal Dynamics (NeurIPS 2025 · similarity 0.81)
- Identifying interactions across brain areas while accounting for individual-neuron dynamics with a Transformer-based variational autoencoder (NeurIPS 2025 · similarity 0.81)
- Brain network science modelling of sparse neural networks enables Transformers and LLMs to perform as fully connected (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.