Research map / Reinforcement learning / RL from feedback
Towards Automated RISC-V Microarchitecture Design with Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an A3C-based multi-objective RL agent that designs RISC-V microarchitectures step by step over a microarchitecture scaling graph, using a preference-conditioned reward and a calibrated lightweight PPA model environment.
Problem
Automated design space exploration for RISC-V microarchitectures to achieve a better performance-power-area (PPA) trade-off, without the unrealistic assumptions of prior data-driven methods and with tighter integration of expert knowledge.
Approach
Formulates microarchitecture design as an MDP whose episode order follows the microarchitecture scaling graph, embeds PPA preference vectors into a single conditioned actor-critic agent (generalized Bellman optimality over preference space, convex coverage set), and evaluates designs with a lightweight environment of XGBoost PPA models calibrated against EDA tools.
Key results
- Average PPA trade-off improvement of 16.03% over prior state-of-the-art approaches with 4.07x higher exploration efficiency (runtime 4130.89 s average).
- Solutions outperform human implementations by up to 2.03x in (Perf x Perf)/(Power x Area).
- Lightweight PPA XGBoost models reach Kendall tau of 0.93, 0.95, 0.94 with roughly 100x speed-up over EDA tools in the loop.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed RL framework instead of prior state-of-the-art approaches | RISC-V Rocket and SonicBOOM microarchitecture design · Rocket and SonicBOOM | PPA trade-off: 16.03% | Helped |
| Use the proposed RL framework for design-space exploration | RISC-V microarchitecture design · Rocket and SonicBOOM | exploration efficiency: 4.07× | Helped |
| Use the proposed RL framework instead of human implementations | RISC-V microarchitecture design · Rocket and SonicBOOM | PPA trade-off: 2.03× | Helped |
| Use the proposed RL method rather than ISCA’14 | Rocket · Rocket | (Perf × Perf)/(Power × Area): 0.364 -> 0.400 | Helped |
| Use the proposed RL method rather than DAC’16 | Small SonicBOOM · Small SonicBOOM | (Perf × Perf)/(Power × Area): 0.035 -> 0.037 | Helped |
| Use the proposed RL method rather than ISCA’14 | Medium SonicBOOM · Medium SonicBOOM | (Perf × Perf)/(Power × Area): 0.048 -> 0.046 | Hurt |
| Use the proposed RL method rather than ISCA’14 | Large SonicBOOM · Large SonicBOOM | (Perf × Perf)/(Power × Area): 0.028 -> 0.031 | Helped |
| Use the proposed RL method rather than DAC’16 | Mega SonicBOOM · Mega SonicBOOM | (Perf × Perf)/(Power × Area): 0.021 -> 0.022 | Helped |
| Use the proposed RL method rather than ISCA’14 | Giga SonicBOOM · Giga SonicBOOM | (Perf × Perf)/(Power × Area): 0.014 -> 0.015 | Helped |
Surprising findings
- Embedding-distance assumptions used by prior data-driven methods do not hold: two microarchitectures at the same embedding distance from a baseline showed very different PPA gaps.
Limitations
- Results evaluated on only Rocket and SonicBOOM configurations and three to ten workloads.
- Agent must be retrained when preference weights change in the baseline formulation; the conditioned network is claimed to mitigate this but is not fully validated.
- Some component relations (e.g., issue buffer size and physical registers) are not in the scaling graph and are decided in a fixed order.
Experimental setup
Datasets: RISC-V Rocket and SonicBOOM design spaces towers, vvadd, spmv RISC-V tests additional RISC-V workloads (median, multiply, dhrystone, etc.)
Benchmarks: RISC-V Rocket design space SonicBOOM Small/Medium/Large/Mega/Giga design space
Metrics: IPC power (mW) area (mm2) Perf/Power Perf/Area (Perf x Perf)/(Power x Area) runtime Kendall tau MAPE
Baselines: Bayesian optimization (ICCAD'21, Bai et al. 2021) AdaBoost-based sampling (DAC'16, Li et al. 2016) ranking-based (ISCA'14, Chen et al. 2014) human efforts (Rocket, SonicBOOM)
Builds on: A3C generalized Bellman optimality (Yang et al. 2019) microarchitecture scaling graph (Eyerman et al. 2009) GAE XGBoost
Closest papers by meaning
- Reinforcement Learning Policy as Macro Regulator Rather than Macro Placer (NeurIPS 2024 · similarity 0.86)
- A Hierarchical Adaptive Multi-Task Reinforcement Learning Framework for Multiplier Circuit Design (ICML 2024 · similarity 0.86)
- Retrieval-Guided Reinforcement Learning for Boolean Circuit Minimization (ICLR 2024 · similarity 0.84)
- AUTOCIRCUIT-RL: Reinforcement Learning-Driven LLM for Automated Circuit Topology Generation (ICML 2025 · similarity 0.83)
- Benchmarking End-To-End Performance of AI-Based Chip Placement Algorithms (NeurIPS 2025 · similarity 0.83)
- Reinforcement Learning within Tree Search for Fast Macro Placement (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.