Research map / Architectures and efficiency / RNNs and normalization

Comparing Linear Regions in ReLU-Type Networks: Theory and Monte Carlo Methods

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Width and depth of ReLU networks

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Characterizes when linear patterns are unique in generic fully connected ReLU and leaky ReLU networks, and uses this with geometry and probability tools to compare linear region counts via Monte Carlo methods.

Problem

Comparing the number of linear regions across different ReLU-type network architectures is hard; theoretical and practical comparisons are needed.

Approach

A complete characterization of the uniqueness of linear patterns for generic fully connected ReLU and leaky ReLU networks, combined with geometric and probabilistic tools, enabling comparison of linear region counts across architectures and Monte Carlo estimation.

Experimental setup

Metrics: number of linear regions

Closest papers by meaning

• The Computational Complexity of Counting Linear Regions in ReLU Neural Networks (NeurIPS 2025 · similarity 0.93)
• Complexity of Linear Regions in Deep Networks (ICML 2019 · similarity 0.91)
• Compelling ReLU Networks to Exhibit Exponentially Many Linear Regions at Initialization and During Training (ICML 2025 · similarity 0.89)
• On the Number of Linear Regions of Convolutional Neural Networks (ICML 2020 · similarity 0.89)
• On the Local Complexity of Linear Regions in Deep ReLU Networks (ICML 2025 · similarity 0.89)
• Expressivity of ReLU-Networks under Convex Relaxations (ICLR 2024 · similarity 0.87)