Research map / Reinforcement learning / Visual and representation RL
Retrieval-Guided Reinforcement Learning for Boolean Circuit Minimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ABC-RL, a retrieval-guided RL method that down-weights a pre-trained policy's recommendations in MCTS for logic synthesis recipe search, using a nearest-neighbor similarity score to set a per-netlist alpha.
Problem
Logic synthesis requires finding good sequences of ABC transformations (synthesis recipes) for new netlists; pre-trained RL agents can misguide MCTS search when test designs differ from training designs (distribution shift).
Approach
Compute a cosine-distance nearest-neighbor similarity between the test AIG's GCN embedding and training AIG embeddings, map it through a sigmoid to alpha, and use pi^(1-alpha) to weight the pre-trained policy in the MCTS UCT term, so novel designs rely more on pure search.
Key results
- ABC-RL achieves a 25.3% geometric mean ADP reduction over resyn2 across 20 test designs, versus 20.7% for MCTS+Learning and 19.8% for MCTS.
- ABC-RL gives a 3.8x geometric mean iso-QoR runtime speed-up over SOTA methods, with up to 9x on individual EPFL designs.
- ABC-RL outperforms MCTS+L+FT (fine-tuning) on all but one netlist, with a 2.66% and 2.40% ADP gain on MCNC and EPFL respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add retrieval-guided alpha tuning to MCTS+Learning | MCTS+Learning · Test circuits, all benchmarks | ADP reduction: 20.7 -> 25.3 | Helped |
| replace BERT recipe encoder with a naive fixed-length encoder (ablation) | ABC-RL · Test circuits, all benchmarks | ADP reduction: 25.3 -> 21.2 | Hurt |
| replace BERT recipe encoder with an LSTM recipe encoder (ablation) | ABC-RL · Test circuits, all benchmarks | ADP reduction: 25.3 -> 22.6 | Hurt |
| continue fine-tuning the pre-trained agent online instead of using retrieval-guided tuning (ablation) | ABC-RL · Test circuits, all benchmarks | ADP reduction: 25.3 -> 23.3 | Hurt |
| train the agent on MCNC only (ablation) | ABC-RL · Test circuits, all benchmarks | ADP reduction: 25.3 -> 21.3 | Hurt |
| train the agent on EPFL arithmetic only (ablation) | ABC-RL · Test circuits, all benchmarks | ADP reduction: 25.3 -> 21.4 | Hurt |
| train the agent on EPFL random control only (ablation) | ABC-RL · Test circuits, all benchmarks | ADP reduction: 25.3 -> 21.7 | Hurt |
Surprising findings
- A pre-trained offline RL agent guiding MCTS underperforms pure MCTS on 11 of 20 designs, despite helping slightly on average.
- Online fine-tuning of the pre-trained agent during search (MCTS+L+FT) did not match ABC-RL on most benchmarks.
Limitations
- Evaluated on only 56 small-to-medium netlists from three benchmark suites with a small training set of 23 circuits.
- Sensitive to hyperparameters threshold and temperature chosen on validation data; ABC-RL degrades on some circuits such as C5 relative to fine-tuning.
- Performance on novel designs depends on the GCN embedding capturing functionality, which the authors note is limited for structurally similar but functionally different AIGs.
Experimental setup
Datasets: MCNC EPFL arithmetic EPFL random control
Benchmarks: MCNC EPFL arithmetic EPFL random control
Metrics: area-delay product reduction vs resyn2 (%) geometric mean ADP reduction iso-QoR runtime speed-up
Baselines: MCTS DRiLLS Online-RL SA+Pred MCTS+Learning MCTS+L+FT
Builds on: AlphaZero MCTS+Learning chip placement fine-tuning DRiLLS SA+Pred
Closest papers by meaning
- A Circuit Domain Generalization Framework for Efficient Logic Synthesis in Chip Design (ICML 2024 · similarity 0.89)
- Towards Next-Generation Logic Synthesis: A Scalable Neural Circuit Generation Framework (NeurIPS 2024 · similarity 0.89)
- AUTOCIRCUIT-RL: Reinforcement Learning-Driven LLM for Automated Circuit Topology Generation (ICML 2025 · similarity 0.87)
- CircuitFusion: Multimodal Circuit Representation Learning for Agile Chip Design (ICLR 2025 · similarity 0.87)
- PCBSchemaGen: Reward-Guided LLM Code Synthesis for Printed Circuit Boards (PCB) Schematic Design with Structured Verification (NeurIPS 2026 · similarity 0.86)
- AnalogCoder: Analog Circuit Design via Training-Free Code Generation (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.