Research map / Large language models / Instruction and fine-tuning
DeepRTL: Bridging Verilog Understanding and Generation with a Unified Representation Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes CodeT5+ on a multi-level Verilog-natural language dataset with curriculum learning to build DeepRTL, a unified model for Verilog understanding and generation, and introduces an understanding benchmark scored with embedding similarity and GPT score.
Problem
Prior LLM work on Verilog focuses almost only on generation and has weak alignment between natural language and Verilog code, and there is no benchmark for Verilog understanding.
Approach
Build a multi-granularity (line, block, module) Verilog-NL dataset annotated with GPT-4 chain-of-thought and human experts, fine-tune CodeT5+ on it with a curriculum that moves from line to module level, and evaluate understanding with embedding similarity and GPT score alongside BLEU/ROUGE.
Key results
- On the understanding benchmark, DeepRTL-220m reaches 0.837 embedding similarity and 0.705 GPT score versus GPT-4's 0.824 and 0.683.
- On Verilog generation, DeepRTL-220m matches o1-preview's 78.06% syntax success rate and beats GPT-4 (63.87%) and its 36.13% functional success rate vs. GPT-4's 27.74% (o1-preview 38.06%).
- Human evaluation gives DeepRTL-220m 78% accuracy on understanding, versus 72% for GPT-4 and 67% for o1-preview.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune CodeT5+-220m on the aligned Verilog and multi-level description dataset with curriculum learning | CodeT5+-220m · Verilog understanding benchmark | Embedding similarity: 0.824 -> 0.837 | Helped |
| Fine-tune CodeT5+-220m on the aligned Verilog and multi-level description dataset with curriculum learning | CodeT5+-220m · Verilog understanding benchmark | GPT Score: 0.683 -> 0.705 | Helped |
| Use curriculum learning instead of direct training without curriculum (ablation) | DeepRTL-220m · Verilog understanding benchmark | Embedding similarity: 0.793 -> 0.837 | Helped |
| Use curriculum learning instead of direct training without curriculum (ablation) | DeepRTL-16b · Verilog understanding benchmark | Embedding similarity: 0.778 -> 0.830 | Helped |
| Use curriculum learning instead of direct training without curriculum (ablation) | DeepRTL-220m · Verilog understanding benchmark | BLEU-4: 11.99 -> 18.66 | Helped |
| Use curriculum learning instead of direct training without curriculum (ablation) | DeepRTL-220m · Verilog understanding benchmark | GPT Score: 0.572 -> 0.705 | Helped |
| Use DeepRTL-220m instead of o1-preview | Verilog generation models · Verilog generation benchmark | Success Rate (syntax): 78.06% -> 78.06% | No clear effect |
| Use DeepRTL-220m instead of o1-preview | Verilog generation models · Verilog generation benchmark | Pass@1 (function): 35.48% -> 32.26% | Hurt |
Surprising findings
- The 220M-parameter DeepRTL outperforms the 16B variant on understanding, which the authors attribute to CodeT5+-220m's paired code-language pre-training.
- BLEU and ROUGE give inconsistent rankings between DeepRTL variants, while embedding similarity and GPT score agree with human evaluation.
Limitations
- Understanding benchmark of only 100 modules and generation evaluated on a small RTLLM-based benchmark.
- About 10% of GPT-4 generated annotations are inaccurate, and designs over 2048 tokens were discarded.
- Generation results are small-sample (5 trials per problem) and mostly reported as success rates on a limited set of designs.
Experimental setup
Datasets: GitHub Verilog modules (open-source) purchased proprietary Verilog IP modules RTLLM-based generation benchmark
Benchmarks: Verilog understanding benchmark (100 modules, introduced) RTLLM-based Verilog generation benchmark
Metrics: BLEU-4 ROUGE-1 ROUGE-2 ROUGE-L embedding similarity GPT score Pass@1 Pass@5 syntax success rate functional success rate
Baselines: GPT-3.5 GPT-4 o1-preview CodeT5+ LLaMA2-70B-Chat deepseek-coder llama-3.2 RTLCoder VeriGen
Builds on: CodeT5+ curriculum learning RTLLM
Closest papers by meaning
- BetterV: Controlled Verilog Generation with Discriminative Guidance (ICML 2024 · similarity 0.88)
- QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generation (NeurIPS 2025 · similarity 0.88)
- CraftRTL: High-quality Synthetic Data Generation for Verilog Code Models with Correct-by-Construction Non-Textual Representations and Targeted Code Repair (ICLR 2025 · similarity 0.87)
- VeriThoughts: Enabling Automated Verilog Code Generation using Reasoning and Formal Verification (NeurIPS 2025 · similarity 0.86)
- VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool (AAAI 2025 · similarity 0.86)
- QiMeng-SALV: Signal-Aware Learning for Verilog Code Generation (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.