AI Paper Atlas

Research map / Continual learning / Online and replay-based

Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks

Lemeng Wu, Bo Liu, Peter Stone, Qiang Liu

NeurIPS 2020 · Continual learning › Online and replay-based · approach: Gradient projection and plasticity preservation

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add brand-new neurons alongside neuron splitting (ablation)Firefly growing a toy RBF network · Toy RBF networktraining lossImproved
Use only neuron splitting, without brand-new neurons (ablation)Firefly growing a toy RBF network · Toy RBF networktraining lossHurt
Use Firefly's greedy selection instead of random searchGrowing a toy RBF network · Toy RBF networktraining lossImproved
Increase the number of brand-new neuron candidates (ablation)Firefly growing a toy RBF network · Toy RBF networktraining lossImproved
Grow depth using Firefly rather than NASHGrowing neural networks · MNISTAccuracy: 99.50 -> 99.59Improved
Grow depth using Firefly rather than AutoGrowGrowing neural networks · MNISTParam (M): 2.3 -> 1.9Improved
Grow depth using Firefly rather than NASHGrowing neural networks · FashionMNISTAccuracy: 94.34 -> 94.48Improved
Grow depth using Firefly rather than AutoGrowGrowing neural networks · FashionMNISTParam (M): 2.3 -> 2.1Improved
Use Firefly for cell-based architecture search instead of DARTS (second order)Cell-based NAS · CIFAR-10Search Time (GPU Days): 4 -> 1.5Improved
Use Firefly for continual learning instead of CPGContinual image classification · CIFAR-100, 20-way splitAvg. Accuracy (20 tasks): 90.75 -> 91.03Improved
Use Firefly for continual learning instead of CPGContinual image classification · CIFAR-100, 20-way splitExtra Growing Epochs: 420 -> 80Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy