Research map / Reinforcement learning / Visual and representation RL
Reinforcement Learning within Tree Search for Fast Macro Placement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EfficientPlace, a bi-level framework combining a global MCTS-style beam/frontier tree search with a PPO-trained U-Net RL agent (guided by wiremask) for sample-efficient macro placement evaluated on ISPD2005.
Problem
Macro placement in chip design via RL is sample-inefficient, requiring many online rollouts or large offline expert data, which is impractical for fast industrial workflows.
Approach
Treats macro placement as a duality of optimization and learning: a global tree search with rolling frontiers directs where a local RL agent (PPO, U-Net policy with wiremask-restricted actions) performs rollouts, and the RL agent is trained during the search to advance it ("learning-inside-optimization").
Key results
- EfficientPlace (1k steps) achieves lowest HPWL on all eight ISPD2005 circuits, averaging about 2.2 hours per chip versus 6.9 hours for WireMask-EA.
- On adaptec2, EfficientPlace reaches 46.79e5 HPWL vs 64.38e5 for WireMask-EA and 67.10e5 for ChiPFormer.
- EfficientPlace achieves second-best results within 500 steps on most chips.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EfficientPlace instead of WireMask-EA (1k) | Macro placement · adaptec1 | HPWL (×105): 6.15±0.15 -> 5.94±0.04 | Helped |
| Use EfficientPlace instead of WireMask-EA (1k) | Macro placement · adaptec2 | HPWL (×105): 64.38±4.43 -> 46.79±1.60 | Helped |
| Use EfficientPlace instead of WireMask-EA (1k) | Macro placement · adaptec3 | HPWL (×105): 58.18±1.04 -> 56.35±0.99 | Helped |
| Use EfficientPlace instead of WireMask-EA (1k) | Macro placement · adaptec4 | HPWL (×105): 59.52±1.71 -> 58.47±1.61 | Helped |
| Use EfficientPlace instead of WireMask-EA (1k) | Macro placement · bigblue1 | HPWL (×105): 2.15±0.01 -> 2.14±0.01 | Helped |
| Use EfficientPlace instead of WireMask-EA (1k) | Macro placement · bigblue2 | HPWL (×105): 5.14±0.06 -> 4.67±0.11 | Helped |
| Use EfficientPlace instead of WireMask-EA (1k) | Macro placement · bigblue3 | HPWL (×105): 59.85±3.39 -> 58.38±0.54 | Helped |
| Use EfficientPlace instead of WireMask-EA (1k) | Macro placement · bigblue4 | HPWL (×105): 77.54±0.67 -> 76.63±1.02 | Helped |
| Add global tree search to the RL agent (ablation) | EfficientPlace · adaptec1 | HPWL | Helped |
| Add U-Net encoder-decoder skip connections (ablation) | EfficientPlace · adaptec1 | HPWL | Helped |
| Use bilinear upsampling instead of deconvolutional layers (ablation) | EfficientPlace · adaptec1 | HPWL | Helped |
| Increase grid resolution (ablation) | EfficientPlace · adaptec1 | HPWL | Helped |
Surprising findings
- Replacing bilinear upsampling with deconvolutional layers, despite more parameters, yields inferior performance.
Limitations
- Optimizes only HPWL without explicitly considering other surrogate metrics such as dataflow and regularity.
- Evaluated only on the ISPD2005 benchmark circuits.
Experimental setup
Datasets: ISPD2005
Benchmarks: ISPD2005 adaptec1 ISPD2005 adaptec2 ISPD2005 adaptec3 ISPD2005 adaptec4 ISPD2005 bigblue1 ISPD2005 bigblue2 ISPD2005 bigblue3 ISPD2005 bigblue4
Metrics: HPWL runtime (hours) congestion (RUDY)
Baselines: GraphPlace DeepPR MaskPlace ChiPFormer SA WireMask-EA
Builds on: MaskPlace ChiPFormer PPO UCT WireMask-EA
Closest papers by meaning
- Chip Placement with Diffusion Models (ICML 2025 · similarity 0.92)
- Reinforcement Learning Policy as Macro Regulator Rather than Macro Placer (NeurIPS 2024 · similarity 0.91)
- EGPlace: An Efficient Macro Placement Method via Evolutionary Search with Greedy Repositioning Guided Mutation (ICML 2025 · similarity 0.88)
- See it to Place it: Evolving Macro Placements with Vision Language Models (NeurIPS 2026 · similarity 0.87)
- A Hierarchical Adaptive Multi-Task Reinforcement Learning Framework for Multiplier Circuit Design (ICML 2024 · similarity 0.86)
- Benchmarking End-To-End Performance of AI-Based Chip Placement Algorithms (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.