Research map / Architectures and efficiency / RNNs and normalization
Empowering Convolutional Neural Nets with MetaSin Activation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace RELU with METASIN activations and train with KD-Bootstrapping | resampling network · Image resampling | PSNR (dB): 33.03 -> 33.26 (P×2) | Helped |
| Add KD-Bootstrapping to METASIN training (ablation) | METASIN resampler · Image resampling P×2 | PSNR (dB): 33.02 -> 33.26 | Helped |
| Enlarge RELU baseline channels (RELUE) to match METASIN compute (ablation) | RELU resampler · Image resampling | PSNR (dB): 33.03 -> 33.04 | No clear effect |
| Apply KD-Bootstrapping to SIREN and MRELU (ablation) | SIREN, MRELU resampler · Image resampling P×2 | PSNR (dB): 31.75 -> 32.76 | Helped |
| Apply KD-Bootstrapping to RELU, MISH and SNAKE (ablation) | RELU/MISH/SNAKE networks · Image resampling | PSNR (dB) | No clear effect |
| Replace RELU with METASIN-5 with KD-B and f_j = j/2 in denoiser | DPCN denoiser · Monte-Carlo denoising | SMAPE: 3.351 -> 3.081 | Helped |
| Replace RELU with METASIN-5 with KD-B and f_j = j/2 in denoiser | DPCN denoiser · Monte-Carlo denoising | 1-MS_SSIM: 3.106 -> 2.733 | Helped |
| Initialize frequencies as f_j = j/2 instead of f_j = j (ablation) | METASIN-5 KD-B · Monte-Carlo denoising | 1-MS_SSIM: 2.757 -> 2.733 | Helped |
| Switch from default frequency init to Glorot initialization (K doubled) (ablation) | METASIN denoiser · Monte-Carlo denoising | 1-MS_SSIM: 2.784 -> 2.838 | Hurt |
| Add KD-Bootstrapping to METASIN-10 training (ablation) | METASIN-10 denoiser · Monte-Carlo denoising | 1-MS_SSIM: 3.066 -> 2.784 | Helped |
| Use MRELU-10 (linear combination of RELUs) instead of RELU (ablation) | DPCN denoiser · Monte-Carlo denoising | 1-MS_SSIM: 3.106 -> 3.397 | Hurt |
| Use MISH instead of RELU (ablation) | DPCN denoiser · Monte-Carlo denoising | 1-MS_SSIM: 3.106 -> 3.011 | Helped |
Closest papers by meaning
- Inducing and Exploiting Activation Sparsity for Fast Inference on Deep Neural Networks (ICML 2020 · similarity 0.87)
- ReLUs Are Sufficient for Learning Implicit Neural Representations (ICML 2024 · similarity 0.87)
- TrAct: Making First-layer Pre-Activations Trainable (NeurIPS 2024 · similarity 0.85)
- PICNN: A Pathway towards Interpretable Convolutional Neural Networks (AAAI 2024 · similarity 0.85)
- Compelling ReLU Networks to Exhibit Exponentially Many Linear Regions at Initialization and During Training (ICML 2025 · similarity 0.85)
- Training the Untrainable: Introducing Inductive Bias via Representational Alignment (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.