Research map / Architectures and efficiency / RNNs and normalization
Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use block-circulant matrices for weights in fully connected and convolutional layers | DNNs on FPGA · MNIST, SVHN, and CIFAR-10 | computational complexity per layer: O($n^2$) to O($n\log n$) | Helped |
| Use block-circulant matrices for weights | DNNs on FPGA · MNIST, SVHN, and CIFAR-10 | storage complexity per layer: O($n^2$) to O($n$) | Helped |
| Use the proposed FPGA framework rather than TrueNorth at similar test accuracy | DNN inference · MNIST, SVHN, and CIFAR-10 | throughput speedup: at least 152X | Helped |
| Use the proposed FPGA framework rather than TrueNorth at similar test accuracy | DNN inference · MNIST, SVHN, and CIFAR-10 | energy efficiency: 71X | Helped |
| Use the proposed FPGA framework rather than the reference FPGA implementation | DNN inference · MNIST, SVHN, and CIFAR-10 | energy efficiency: at least 31X | Helped |
| Decouple FFT and IFFT operations by reusing input FFTs and postponing IFFT until after summation | Block-circulant fully connected layer inference · Fully connected layers | number of FFT calculations: from $p\cdot q$ to $q$ | Helped |
| Move IFFT after summing the frequency-domain products | Block-circulant fully connected layer inference · Fully connected layers | number of IFFT calculations: reduced by $q$ times | Helped |
| Keep the whole DNN model in on-chip block memory | FPGA DNN implementation · MNIST, SVHN, and CIFAR-10 | energy efficiency | Helped |
| Use variational inference-based Bayesian training | Small-to-medium neural networks · Small-data training | accuracy and robustness | Helped |
Closest papers by meaning
- ShiftAddNet: A Hardware-Inspired Deep Network (NeurIPS 2020 · similarity 0.86)
- DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks (ICML 2022 · similarity 0.86)
- Towards training digitally-tied analog blocks via hybrid gradient computation (NeurIPS 2024 · similarity 0.84)
- An Empirical Study of Distributed Deep Learning Training on Edge (Student Abstract) (AAAI 2024 · similarity 0.83)
- JAQ: Joint Efficient Architecture Design and Low-Bit Quantization with Hardware-Software Co-Exploration (AAAI 2025 · similarity 0.83)
- Accelerating neural network training: An analysis of the AlgoPerf competition (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.