Research map / Vision-language and multimodal / Multimodal LLMs

CircuitSense: A Hierarchical Circuit System Benchmark Bridging Visual Comprehension and Symbolic Reasoning in Engineering Design Process

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces CircuitSense, a hierarchical benchmark (8,006 problems) of visual circuit and block-diagram understanding with symbolic equation ground truth, and evaluates eight MLLMs across perception, analysis and design tasks.

Problem

MLLMs' ability to extract symbolic mathematical models (transfer functions, nodal equations) from circuit schematics and block diagrams across abstraction levels is untested; existing visual circuit benchmarks focus on recognition or shallow calculation.

Approach

A benchmark combining curated textbook/course problems with synthetic grid-based circuit schematics (SPICE/Lcapy-validated) and random block diagrams (Mason's gain formula) that have auto-derived symbolic ground truth, scored with SymPy-based equivalence checking, organized over six hierarchy levels and three task categories.

Key results

• Gemini-2.5-Pro reaches 19.06% overall on synthetic symbolic-derivation problems, while closed-source models exceed 85% on some perception subtasks.
• Gemini-2.5-Pro achieves 80.71% multiple-choice and 70.32% open-ended accuracy on curated Analysis problems, versus 19.06% on synthetic circuits.
• On design tasks Gemini-2.5-Pro scores 36.38% schematic-level, 67.27% block-level, and 51.35% hierarchical.

Surprising findings

• Open-ended accuracy on curated problems collapses for most models relative to multiple choice, indicating reliance on answer elimination.
• Output impedance derivation is a major bottleneck (8% for Gemini-2.5-Pro) despite high component identification accuracy (97%).

Limitations

• Synthetic pipeline covers only transfer function and nodal analysis; other analysis types are absent.
• Synthetic circuits are limited to about 12-15 components due to symbolic computation cost.
• Evaluation depends on proprietary or preview model versions and an LLM judge (Gemini-2.5-Flash).

Experimental setup

Datasets: curated textbook and university course problems AnalogGenie ZeroSim synthetic schematic and block diagram problems

Benchmarks: CircuitSense

Metrics: accuracy symbolic equivalence (SymPy, numerical validation)

Baselines: GPT-4o GPT-4o-mini Claude-Sonnet-4 InternVL3-78B Qwen2.5-VL-72B-Instruct GLM-4.5V Gemma-3-27B

Builds on: MAPS Lcapy Mason's gain formula SPICE

Closest papers by meaning

• AutoVSR: Automatic Visual-to-Symbolic Reasoning for Symbolic Expression Generation from Circuit Schematic (ICML 2026 · similarity 0.89)
• EEE-Bench: A Comprehensive Multimodal Electrical And Electronics Engineering Benchmark (CVPR 2025 · similarity 0.87)
• Math Blind: Failures in Diagram Understanding Undermine Reasoning in MLLMs (ICLR 2026 · similarity 0.86)
• CircuitFusion: Multimodal Circuit Representation Learning for Agile Chip Design (ICLR 2025 · similarity 0.85)
• Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition (ICLR 2025 · similarity 0.85)
• MAPS: Advancing Multi-Modal Reasoning in Expert-Level Physical Science (ICLR 2025 · similarity 0.85)