Research map / Architectures and efficiency / CNN design
Convolutional Channel-Wise Competitive Learning for the Forward-Forward Algorithm
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a channel-wise competitive learning loss (CwC) and a CFSE block with grouped convolutions to train CNNs layer-wise with the Forward-Forward framework, removing the need for negative data and reaching lower error than prior FF methods on image classification.
Problem
The Forward-Forward algorithm relies on negative data generation, converges slowly, and performs poorly on complex image tasks compared with backpropagation.
Approach
A layer-wise channel-wise competitive loss (CwC) that splits conv channels into per-class groups and applies softmax cross-entropy over per-class goodness scores, removing negative data; combined with a CFSE block (standard conv followed by channel-grouped conv) and an interleaved layer training schedule.
Key results
- CFSE_CwC+Sf reaches 0.58% test error on MNIST, 7.69% on Fashion-MNIST, 21.89% on CIFAR-10 and 48.77% on CIFAR-100, outperforming FF-based baselines.
- Reported CaFo CIFAR-10 error of 32.57% after 5000 epochs versus 21.89% for the proposed model trained for 40 epochs (50 with Sf predictor).
- Ablation shows CwC loss gives the largest gains over the original FF goodness loss, e.g. CIFAR-10 error drops from ~46% (FF-FC rep) to ~23% with CwC+Gd.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CwC loss instead of PvN loss (ablation) | FF-CNN with Goodness predictor · CIFAR-10 | test error (%): 33.79 ± 0.84 -> 23.16 ± 0.16 | Helped |
| use CwC loss instead of PvN loss (ablation) | FF-CNN with Goodness predictor · Fashion-MNIST | test error (%): 12.73 ± 0.46 -> 7.88 ± 0.53 | Helped |
| add CFSE blocks (ablation) | CwC model with Softmax predictor · CIFAR-10 | test error (%): 22.54 ± 0.16 -> 21.89 ± 0.44 | Helped |
| add CFSE blocks (ablation) | CwC model with Softmax predictor · MNIST | test error (%): 0.59 ± 0.09 -> 0.58 ± 0.08 | No clear effect |
| use Softmax predictor instead of Goodness predictor (ablation) | CFSE_CwC · CIFAR-10 | test error (%): 24.43 ± 0.34 -> 21.89 ± 0.44 | Helped |
| use Global Averaging predictor instead of Softmax predictor (ablation) | CFSE_CwC · CIFAR-10 | test error (%): 21.89 ± 0.44 -> 27.25 ± 0.49 | Hurt |
| use CFSE_CwC+Sf instead of BP-CNN | image classification · MNIST | test error (%): 0.56 -> 0.58 | Hurt |
| use CFSE_CwC+Sf instead of BP-CNN | image classification · Fashion-MNIST | test error (%): 6.75 -> 7.69 | Hurt |
| use CFSE_CwC+Sf instead of SoftHebb | image classification · CIFAR-10 | test error (%): 19.7 -> 21.89 | Hurt |
Surprising findings
- Softmax predictor outperforms goodness and global-averaging predictors across datasets, with the GA predictor being cheapest but least accurate.
Limitations
- Still lags behind backpropagation on more complex datasets.
- Scalability to larger numbers of classes is untested; class grouping is only a preliminary workaround.
- Interleaved layer training strategy is described as preliminary and heuristic.
Experimental setup
Datasets: MNIST Fashion-MNIST CIFAR-10 CIFAR-100
Benchmarks: MNIST Fashion-MNIST CIFAR-10 CIFAR-100
Metrics: test error (%) training epochs
Baselines: FF-FC (rep*) CaFo PFF-RNN SoftHebb DFA DRTP FFF BP-CNN (rep*)
Builds on: Forward-Forward algorithm Cascaded Forward SoftHebb Direct Feedback Alignment
Closest papers by meaning
- Scaling Forward Gradient With Local Losses (ICLR 2023 · similarity 0.86)
- Dynamic Channel Pruning: Feature Boosting and Suppression (ICLR 2019 · similarity 0.86)
- Layer Free-Riding in Forward-Forward Networks: Real, Repairable, but Not Accuracy-Dominant (NeurIPS 2026 · similarity 0.85)
- The Price of Locality: Why Forward-Forward Underperforms Backpropagation? (NeurIPS 2026 · similarity 0.85)
- Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers (ICLR 2018 · similarity 0.85)
- Forward Learning of Graph Neural Networks (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.