Research map / Large language models / Agents, code and math
QiMeng-SALV: Signal-Aware Learning for Verilog Code Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes QiMeng-SALV, which verifies output signals of generated Verilog modules against reference modules, extracts per-signal code segments via AST, and applies a signal-aware DPO loss on those segments to train a 7B Verilog generator.
Problem
RL for Verilog code generation lacks meaningful functional rewards because module-level correctness is rarely achieved by SFT models, so most generated samples provide no usable preference signal.
Approach
Verify correctness per output signal by simulation against a reference module, use AST dependency analysis to isolate the code segments of correct signals (even in partially incorrect modules), and compute the DPO loss only over those signal-level segments.
Key results
- On RTLLM v1.1, QiMeng-SALV (7B) reaches 62.6% pass@1 and 75.1% pass@5, matching DeepSeek-V3 (671B, 62.0% pass@1) and surpassing it at pass@10 (81.1% vs 72.4%).
- On VerilogEval1.0 Machine, QiMeng-SALV reaches 81.4 pass@1, above DeepSeek-V3's 77.6; Human pass@1 is 60.4.
- Ablation on RTLLM v1.1: signal-aware DPO raises pass@1 from 48.3 (SFT) to 62.6, while training on mixed partial-correct data without filtering gives 55.3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply QiMeng-SALV instead of Deepseek v3 | Verilog generation · RTLLM v1.1 | pass@1: 62.0 -> 62.6 | Helped |
| apply QiMeng-SALV instead of Deepseek v3 | Verilog generation · RTLLM v1.1 | pass@10: 72.4 -> 81.1 | Helped |
| apply QiMeng-SALV instead of Deepseek v3 | Verilog generation · RTLLM v2.0 | pass@1: 59.1 -> 62.0 | Helped |
| apply QiMeng-SALV instead of CodeV (Qwen2.5) | Verilog generation · RTLLM v1.1 | pass@1: 39.3 -> 62.6 | Helped |
| apply QiMeng-SALV instead of CodeV (Qwen2.5) | Verilog generation · RTLLM v2.0 | pass@1: 41.0 -> 62.0 | Helped |
| use partial correct datasets only instead of complete correct datasets only (ablation) | signal-aware DPO training · RTLLM v1.1 | pass@1: 57.4 -> 56.0 | Hurt |
| add partial correct datasets without filtering incorrect signals to complete correct datasets (ablation) | signal-aware DPO training · RTLLM v1.1 | pass@1: 57.4 -> 55.3 | Hurt |
| filter incorrect signals in partial-correct-only training (ablation) | signal-aware DPO training · RTLLM v1.1 | pass@1: 56.0 -> 56.6 | Helped |
| filter incorrect signals when training on complete and partial correct datasets (ablation) | signal-aware DPO training · RTLLM v1.1 | pass@1: 55.3 -> 62.6 | Helped |
| add SFT (ablation) | Qwen2.5 base model · RTLLM v1.1 | pass@1: 30.1 -> 48.3 | Helped |
| add signal-aware DPO after SFT (ablation) | Qwen2.5 with SFT · RTLLM v1.1 | pass@1: 48.3 -> 62.6 | Helped |
| increase signal-aware DPO preference-dataset size (ablation) | QiMeng-SALV · RTLLM benchmark | pass@1 | Helped |
Surprising findings
- Naively adding partially correct modules to DPO training (without filtering incorrect signals) degraded performance relative to training on complete correct modules alone.
Limitations
- Functional verification depends on the reference module in the training data being correct; errors in references can yield wrong correctness labels.
- Relies on a high-quality training dataset, which the paper does not address how to obtain.
Experimental setup
Datasets: CodeV dataset (135k cleaned from 165k GitHub Verilog modules)
Benchmarks: VerilogEval1.0 VerilogEval2.0 RTLLM v1.1 RTLLM v2.0
Metrics: pass@1 pass@5 pass@10
Baselines: GPT-4o DeepSeek-V3 CodeV Origen VeriPrefer RTLCoder VeriSeek Qwen2.5 Coder Instruct DeepSeek Coder CodeQwen1.5
Builds on: DPO CodeV Yosys LoRA
Closest papers by meaning
- QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generation (NeurIPS 2025 · similarity 0.90)
- BetterV: Controlled Verilog Generation with Discriminative Guidance (ICML 2024 · similarity 0.88)
- VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool (AAAI 2025 · similarity 0.87)
- VeriThoughts: Enabling Automated Verilog Code Generation using Reasoning and Formal Verification (NeurIPS 2025 · similarity 0.86)
- CraftRTL: High-quality Synthetic Data Generation for Verilog Code Models with Correct-by-Construction Non-Textual Representations and Targeted Code Repair (ICLR 2025 · similarity 0.86)
- SymRTLO: Enhancing RTL Code Optimization with LLMs and Neuron-Inspired Symbolic Reasoning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.