AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Learning Specialized Activation Functions With the Piecewise Linear Unit

Yucong Zhou, Zezhou Zhu, Zhao Zhong

ICCV 2021 · Architectures and efficiency › RNNs and normalization · approach: Width and depth of ReLU networks · 14 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add statistic-based realignment instead of fix-init-10 (channel-wise) (ablation)PWLU on ResNet-18 · ImageNettop-1 accuracy: 72.19 -> 72.57Improved
add statistic-based realignment instead of fix-init-3 (channel-wise) (ablation)PWLU on MobileNet-V3 · ImageNettop-1 accuracy: 68.98 -> 69.65Improved
add statistic-based realignment instead of fix-init-10 (layer-wise) (ablation)PWLU on ResNet-18 · ImageNettop-1 accuracy: 71.84 -> 72.22Improved
add statistic-based realignment instead of fix-init-10 (channel-wise) (ablation)PWLU on MobileNet-V3 · ImageNettop-1 accuracy: 68.20 -> 69.65Improved
use fixed hand-picked boundaries instead of ReLU baseline (layer-wise fix-init-3) (ablation)PWLU on ResNet-18 · ImageNettop-1 accuracy: 71.34 -> 71.57Improved
increase intervals from 4 to 8 (ablation)PWLU on MobileNet-V3 · ImageNettop-1 accuracy: 68.70 -> 69.55Improved
increase intervals from 16 to 20 (ablation)PWLU on MobileNet-V3 · ImageNettop-1 accuracy: 69.65 -> 69.37Hurt
increase intervals from 16 to 20 (ablation)PWLU on ResNet-18 · ImageNettop-1 accuracy: 72.57 -> 72.43Hurt
replace Swish with PWLUResNet-18 · ImageNettop-1 accuracy: 71.61 -> 72.57Improved
replace Swish with PWLUMobileNet-V3 · ImageNettop-1 accuracy: 67.93 -> 69.65Improved
replace Swish with PWLUMask R-CNN ResNet-50-FPN · COCO val2017AP bbox: 40.05 -> 40.48Improved
replace Swish with PWLURetinaNet ResNet-50-FPN · COCO val2017AP bbox: 37.82 -> 38.31Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy