Research map / Architectures and efficiency / CNN design
Convolutional Rectifier Networks as Generalized Tensor Decompositions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace linear activation and product pooling with ReLU activation and max pooling (convolutional rectifier network) | shallow ConvNet · theoretical universality analysis | universality (ability to realize any grid tensor) | Helped |
| Use ReLU activation with average (sum) pooling instead of max pooling | shallow and deep ConvNets · theoretical universality analysis | universality (ability to realize any grid tensor) | Hurt |
| Expand conv receptive field from 1x1 to w x h (less than half the feature maps) with ReLU and average pooling | shallow ConvNet with average pooling · theoretical universality analysis | universality | No clear effect |
| Expand conv receptive field to cover entire spatial extent (fully-connected) with ReLU activation | shallow network · theoretical universality analysis | universality | Helped |
| Use deep ConvNet (L=log2 N) with ReLU and max pooling instead of shallow network, for depth efficiency | convolutional rectifier networks · theoretical depth efficiency | minimal shallow width Z required (rank bound) | Helped |
| Randomize weights of deep ReLU max-pooling ConvNet | convolutional rectifier networks depth efficiency · theoretical depth efficiency | completeness of depth efficiency | Mixed |
Closest papers by meaning
- Understanding Deep Neural Networks with Rectified Linear Units (ICLR 2018 · similarity 0.87)
- Exploring Unexplored Tensor Network Decompositions for Convolutional Neural Networks (NeurIPS 2019 · similarity 0.87)
- Boosting Dilated Convolutional Networks with Mixed Tensor Decompositions (ICLR 2018 · similarity 0.87)
- Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods (NeurIPS 2024 · similarity 0.86)
- Graph Neural Networks and Arithmetic Circuits (NeurIPS 2024 · similarity 0.85)
- A Fast Convoluted Story: Scaling Probabilistic Inference for Integer Arithmetics (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.