Research map / Reinforcement learning / Visual and representation RL
Reinforcement Learning Policy as Macro Regulator Rather than Macro Placer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MaskRegulate, an RL policy (PPO) that refines existing macro placements instead of placing macros from scratch, adding a regularity reward, and evaluates on ICCAD 2015 and ISPD 2005 with commercial PPA tools.
Problem
RL-based macro placement suffers from long training, poor generalization, and no PPA guarantees, largely because placing from scratch gives limited state information and sparse, inaccurate rewards; prior methods also ignore regularity.
Approach
Reformulate RL placement as a regulator that iteratively adjusts macros of an existing layout (from any placer) rather than a placer building the layout from scratch, with full-layout state information, a WireMask normalized to [-1,1], a RegularMask encoding peripheral regularity, and a reward mixing HPWL reduction and regularity improvement.
Key results
- On ICCAD 2015 superblue1, MaskRegulate vs MaskPlace improves routing wirelength by 17.08%, horizontal/vertical congestion overflow by 73.08%/38.81%, and TNS by 37.89%.
- MaskRegulate achieves the best average rank on global HPWL (1.25) and regularity (1) across the eight ICCAD 2015 chips.
- Regulating placements from MaskPlace, AutoDMP, or WireMask-EA with the pretrained regulator improves regularity on all four unseen chips and HPWL on three.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace MaskPlace's placer formulation with regulator refinement (ablation) | MaskPlace · ICCAD 2015 superblue1 | Global HPWL: 7.93 $\pm$ 0.06 -> 5.58 $\pm$ 0.05 | Helped |
| replace MaskPlace's placer formulation with regulator refinement (ablation) | MaskPlace · ICCAD 2015 superblue3 | Global HPWL: 9.00 $\pm$ 0.17 -> 8.01 $\pm$ 0.03 | Helped |
| replace MaskPlace's placer formulation with regulator refinement (ablation) | MaskPlace · ICCAD 2015 superblue4 | Global HPWL: 5.28 $\pm$ 0.03 -> 4.31 $\pm$ 0.01 | Helped |
| replace MaskPlace's placer formulation with regulator refinement (ablation) | MaskPlace · ICCAD 2015 superblue5 | Global HPWL: 9.81 $\pm$ 0.03 -> 7.89 $\pm$ 0.02 | Helped |
| add reward normalization (ablation) | MaskRegulate · ICCAD 2015 superblue3 | Global HPWL: 6.71 $\pm$ 0.03 -> 7.05 $\pm$ 0.03 | Hurt |
| add reward normalization (ablation) | MaskRegulate · ICCAD 2015 superblue3 | Regularity: 3.59 $\pm$ 0.01 -> 3.54 $\pm$ 0.00 | Helped |
| add RegularMask and regularity-based reward to a placer trained from scratch (ablation) | MaskPlace · ICCAD 2015 superblue1 | Regularity: 4.40 $\pm$ 0.06 -> 3.87 $\pm$ 0.06 | Helped |
| add RegularMask and regularity-based reward to a placer trained from scratch (ablation) | MaskPlace · ICCAD 2015 superblue1 | Global HPWL: 7.93 $\pm$ 0.06 -> 7.44 $\pm$ 0.08 | Helped |
| add RegularMask and regularity-based reward to a placer trained from scratch (ablation) | MaskPlace · ICCAD 2015 superblue10 | Global HPWL: 10.94 $\pm$ 0.21 -> 10.90 $\pm$ 0.30 | No clear effect |
| add regulator refinement formulation to MaskPlace + RegularMask (ablation) | MaskPlace + RegularMask · ICCAD 2015 superblue1 | Global HPWL: 7.44 $\pm$ 0.08 -> 5.77 $\pm$ 0.05 | Helped |
| add regulator refinement formulation to MaskPlace + RegularMask (ablation) | MaskPlace + RegularMask · ICCAD 2015 superblue10 | Global HPWL: 10.90 $\pm$ 0.30 -> 11.23 $\pm$ 0.38 | Hurt |
| regulate placements from MaskPlace | MaskPlace · ICCAD 2015 superblue10 | Global HPWL: 10.55 -> 11.20 | Hurt |
Surprising findings
- MaskRegulate performs better on some unseen chips (e.g., superblue16) than on chips it was trained on in terms of global HPWL.
- Without normalization, the HPWL scale dominates the reward, so the regulator does not prioritize regularity.
Limitations
- Does not model module aspect ratio and area factors.
- Ignores global wirelength and timing metrics during training.
- Does not use advanced transformer architectures to improve regulator generalization.
Experimental setup
Datasets: ICCAD 2015 benchmark ISPD 2005 benchmark
Benchmarks: ICCAD 2015 ISPD 2005
Metrics: global HPWL regularity routed wirelength routed horizontal and vertical congestion overflow worst negative slack total negative slack number of violation points
Baselines: DREAMPlace AutoDMP WireMask-EA MaskPlace ChiPFormer
Builds on: MaskPlace WireMask-BBO DREAMPlace PPO
Closest papers by meaning
- Chip Placement with Diffusion Models (ICML 2025 · similarity 0.92)
- Reinforcement Learning within Tree Search for Fast Macro Placement (ICML 2024 · similarity 0.91)
- Towards Automated RISC-V Microarchitecture Design with Reinforcement Learning (AAAI 2024 · similarity 0.86)
- Benchmarking End-To-End Performance of AI-Based Chip Placement Algorithms (NeurIPS 2025 · similarity 0.86)
- EGPlace: An Efficient Macro Placement Method via Evolutionary Search with Greedy Repositioning Guided Mutation (ICML 2025 · similarity 0.86)
- Retrieval-Guided Reinforcement Learning for Boolean Circuit Minimization (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.