Research map / Learning theory and algorithms / Prediction and decision losses
HOPE: High-Order Polynomial Expansion of Black-Box Neural Networks
Tingxiong Xiao, Weihang Zhang, Yuxiao Cheng, Jinli Suo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HOPE instead of Autograd for 10th-order derivatives | 10-layer MLP, 1-D input · 1-D input MLP, order 10 | time (s): 5.4099 -> 0.1942 | Improved |
| use HOPE instead of Autograd for 5th-order derivatives | 10-layer MLP, 3-D input · 3-D input MLP, order 5 | time (s): 6.9751 -> 0.0711 | Improved |
| use HOPE instead of Autograd for 6th-order unmixed derivatives | MNIST 01 classifier · MNIST 01 classifier, order 6 | time (s): 1129.2 -> 0.0479 | Improved |
| use HOPE where Autograd runs out of memory | MNIST 01 classifier · MNIST 01 classifier, order 7-10 | memory: OOM / 76+ -> 0.1000 / 0.30 | Improved |
| replace network with 3rd-order Taylor polynomial | level controller · batch 4096 inference | inference time (ms): 121.98 -> 0.0324 | Improved |
| replace network with 3rd-order Taylor polynomial | level controller · model size | bytes: 44,671 -> 160 | Improved |
| replace perturbation analysis with HOPE heat map | MNIST classifier · heat map generation | time (s): 0.342 -> 0.002 | Improved |
| reduce weight scale w0 from 1.0 to 0.01 (ablation) | MLP · MLP width 512 depth 5 | 10th-order derivative ratio: 1.39e-01 -> 1.31e-19 | Improved |
| increase weight scale w0 from 1.0 to 100 (ablation) | MLP · MLP width 512 depth 5 | 10th-order derivative ratio: 1.39e-01 -> 1.23e+21 | Hurt |
| use HOPE for local approximation of ReLU and max-pool networks | HOPE · networks with ReLU or max pooling | derivative order obtained | No clear effect |
Closest papers by meaning
- TMPNN: High-Order Polynomial Regression Based on Taylor Map Factorization (AAAI 2024 · similarity 0.88)
- Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks (ICLR 2020 · similarity 0.86)
- Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials (NeurIPS 2022 · similarity 0.86)
- Learning Hierarchical Polynomials with Three-Layer Neural Networks (ICLR 2024 · similarity 0.85)
- Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit (NeurIPS 2024 · similarity 0.85)
- Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operators (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
