Research map / Architectures and efficiency / RNNs and normalization
Activation with Intrinsic-Extrinsic Consensus
Tian Qiu, Zunlei Feng, Yang Gao, Bingde Hu, Yi Gao, Mingli Song
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AIEC activation instead of ReLU | ViT-Tiny · CIFAR-100 | top-1 accuracy: 65.7 -> 71.1 | Improved |
| use AIEC activation instead of ReLU | TNT-Small · ImageNet-1K | top-1 accuracy: 73.4 -> 78.1 | Improved |
| add gatekeeping loss (auxiliary loss alone) (ablation) | AIEC on ViT-Tiny · CIFAR-100 | top-1 accuracy: 66.7 -> 71.1 | Improved |
| use intrinsic-extrinsic consensus rather than intrinsic only (ablation) | AIEC on ViT-Tiny · CIFAR-100 | top-1 accuracy: 69.7 -> 71.1 | Improved |
| use intrinsic-extrinsic consensus rather than extrinsic only (ablation) | AIEC on ViT-Tiny · CIFAR-100 | top-1 accuracy: 68.5 -> 71.1 | Improved |
| use consensus (intersection) rather than union of assessments (ablation) | AIEC on ViT-Tiny · CIFAR-100 | top-1 accuracy: 69.6 -> 71.1 | Improved |
| suppress only irrelevant channels instead of all channels (ablation) | AIEC on ViT-Tiny · CIFAR-100 | top-1 accuracy: 65.8 -> 71.1 | Improved |
| use AIEC instead of L0 regularization | ViT-Tiny · CIFAR-100 | top-1 accuracy: 67.2 -> 71.1 | Improved |
| use AIEC instead of SENet | ResNet-18 · CIFAR-100 | top-1 accuracy: 75.6 -> 77.0 | Improved |
| use AIEC for graph anomaly detection instead of ReLU | GraphSAGE · T-Finance | AUC: 84.5 -> 90.9 | Improved |
| use AIEC for graph anomaly detection instead of ReLU | GCN · Weibo21 | AUC: 98.5 -> 98.6 | No clear effect |
| use AIEC instead of GELU | ViT-Tiny training · ImageNet 224x224 batch 1024 | training GPU memory: 28.3 -> 32.5 GiB | No clear effect |
Closest papers by meaning
- Dual-Perspective Activation: Efficient Channel Denoising via Joint Forward-Backward Criterion for Artificial Neural Networks (NeurIPS 2024 · similarity 0.86)
- Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization (ICLR 2024 · similarity 0.82)
- Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks (NeurIPS 2025 · similarity 0.81)
- Exploring the Benefit of Activation Sparsity in Pre-training (ICML 2024 · similarity 0.81)
- Exploiting Activation Sparsity with Dense to Dynamic-k Mixture-of-Experts Conversion (NeurIPS 2024 · similarity 0.81)
- Task-Specific Data Selection for Instruction Tuning via Monosemantic Neuronal Activations (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
