Research map / Architectures and efficiency / Attention and transformers
Circuit Transformer: A Transformer That Preserves Logical Equivalence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Circuit Transformer, a Transformer that generates Boolean circuits token-by-token with a cutoff-based masking layer so every output is strictly logically equivalent to the target function, and optimizes circuit size via MCTS over an MDP formulation.
Problem
Neural generative models for circuit realization occasionally produce circuits that are not logically equivalent to the target Boolean function, which is unacceptable for exact digital design.
Approach
A top-down sequential circuit encoding with cutoff properties (inheritable, incrementally testable via three-valued logic truth tables) lets a masking layer block any next token that would break equivalence, so the Transformer is trained normally with cross-entropy but guarantees equivalence at inference; the generation is also cast as an MDP for size minimization with MCTS.
Key results
- Circuit Transformer with MCTS (K=100) reaches 0.17% unsuccessful and average size 13.73 on random circuits vs 14.56 for Resyn2 ground truth, with zero equivalence violations.
- On IWLS FFWs, Circuit Transformer (K=100) gets 2.63% unsuccessful and average size 16.73 vs AIGER Transformer with beam 128 at 4.23% and 17.12.
- Without the masking layer, the model had 2.59% violation cases on random circuits; with the masking layer, violations drop to 0%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add tree positional encoding (ablation) | Circuit Transformer · Random circuits | average circuit size: 15.02 -> 14.79 | Helped |
| add tree positional encoding (ablation) | Circuit Transformer · IWLS FFWs | average circuit size: 17.33 -> 17.17 | Helped |
| add equivalence masking (ablation) | Circuit Transformer · Random circuits | violation cases: 2.59% -> 0.00% | Helped |
| add equivalence masking (ablation) | Circuit Transformer · IWLS FFWs | violation cases: 6.54% -> 0.00% | Helped |
| increase MCTS playouts from the default model to K=10 | Circuit Transformer · Random circuits | average circuit size: 14.79 -> 14.02 | Helped |
| increase MCTS playouts from the default model to K=100 | Circuit Transformer · IWLS FFWs | average circuit size: 17.17 -> 16.73 | Helped |
Surprising findings
- With MCTS playouts, the Circuit Transformer sometimes produces circuits more compact than the Resyn2 ground truth it was trained on.
Limitations
- Unsuccessful cases remain from exceeding the maximum sequence length of 200.
- Unfolding DAGs into trees can produce long sequences; compact representation left as future work.
- Only AND/NOT gate basis and 8-input functions evaluated; truth-table-based checks scale as 2^N.
Experimental setup
Datasets: randomly generated 8-input 2-output circuits (15M, Resyn2 labels) IWLS 2023 benchmark fanout-free windows
Benchmarks: random circuits (synthetic test split, 10,240 circuits) IWLS 2023 FFWs (10,240 sampled)
Metrics: unsuccessful cases percentage equivalence violation percentage average circuit size percentage strictly smaller than Resyn2 average time per circuit
Baselines: Boolean Chain Transformer AIGER Transformer beam search variants (beam size 16, 128) Resyn2
Builds on: Transformer encoder-decoder PUCT (AlphaZero-style MCTS) ABC Resyn2 optimization flow AIGER format backtracking with cutoff properties
Closest papers by meaning
- Towards Next-Generation Logic Synthesis: A Scalable Neural Circuit Generation Framework (NeurIPS 2024 · similarity 0.86)
- Finding Transformer Circuits With Edge Pruning (NeurIPS 2024 · similarity 0.86)
- Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition (ICLR 2025 · similarity 0.86)
- Learning Linear Attention in Polynomial Time (NeurIPS 2025 · similarity 0.84)
- Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gates (NeurIPS 2025 · similarity 0.84)
- AnalogGenie: A Generative Engine for Automatic Discovery of Analog Circuit Topologies (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.