Research map / Large language models / Agents, code and math
Dataflow-Guided Neuro-Symbolic Language Models for Type Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NESTER, a neuro-symbolic framework that decomposes Python type inference into a high-level program of dataflow/control-flow API calls, executed by an interpreter that mixes static analysis with small local LMs (Code Llama 7B, Llama 3 8B).
Problem
Type inference with small locally deployable LMs struggles to reason about control and data flow in code, limiting accuracy for Python type annotations.
Approach
Generate a high-level program of modular API calls (condition evaluation and type inference entry points) from the code via LM prompting, then interpret it with a program interpreter that traverses a coarse-grained dataflow graph, using a type recognizer for static types and LMs for the rest, without increasing model size.
Key results
- NESTER reaches 70.7% Top-1 Exact Match overall on ManyTypes4Py, 3.6% above TypeGen and 18.3% above HiTyper.
- On complex annotations, NESTER reports 51.0% for typing.Optional and 16.7% for typing.Union, versus TypeGen's 22.7% and 10.9%.
- NESTER-CL (Code Llama 7B backbone) achieves 81.5% Top-1 Match to Parametric overall, above Type4Py's 80.2%, though it trails Type4Py on Exact Match.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use NESTER-CL instead of TypeGen-CL | Code Llama 7B for type inference · ManyTypes4Py | Top-1 Exact Match (All): 67.1 -> 70.7 | Helped |
| use NESTER-CL instead of TypeGen-CL | Code Llama 7B for type inference · ManyTypes4Py | Top-1 Match to Parametric (All): 79.0 -> 81.5 | Helped |
| use NESTER-CL instead of Type4Py | type inference for arguments · ManyTypes4Py arguments | Top-1 Exact Match: 66.5 -> 70.3 | Helped |
| use NESTER-CL instead of Type4Py | type inference for return types · ManyTypes4Py return types | Top-1 Exact Match: 56.1 -> 60.0 | Helped |
| replace matching-type demonstrations with similar examples having different type answers (ablation) | TypeGen with Code Llama 7B · ManyTypes4Py | accuracy: 72.5% -> 64% | Hurt |
| remove demonstration examples (ablation) | TypeGen with Code Llama 7B · ManyTypes4Py | accuracy: 72.5% -> 57.2% | Hurt |
Surprising findings
- TypeGen's accuracy drops from 72.5% to 64% when demonstrations with identical answers are replaced by similar-code examples, suggesting LMs rely on ICL answer leakage rather than reasoning.
- NESTER lags Type4Py on overall Top-1 Exact Match, attributed to Type4Py's strength on short code segments.
Limitations
- Restricted applicability to larger LLMs; the approach was only instantiated with 7B-8B models.
- Extending to other languages such as Java requires substantial redesign.
- Symbolic rule coverage is low, especially for argument and return types (0.1% and 3.4% total coverage).
Experimental setup
Datasets: ManyTypes4Py
Benchmarks: ManyTypes4Py
Metrics: Top-1 Exact Match Top-3 Exact Match Top-5 Exact Match Match to Parametric human evaluation (correctness, explainability, factuality, c
Baselines: HiTyper TypeBERT TypeWriter UniXcoder Type4Py InCoder CodeT5 TypeGen Code Llama 7B Llama 3 8B
Builds on: TypeGen VISPROG Type4Py
Closest papers by meaning
- TypyBench: Evaluating LLM Type Inference for Untyped Python Repositories (ICML 2025 · similarity 0.89)
- CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidance (ICML 2025 · similarity 0.87)
- Cross-Dialect Generalization Without Retraining: Benchmarks and Evaluation of Schema-Derived Constrained Decoding for MLIR (NeurIPS 2026 · similarity 0.84)
- InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct (AAAI 2025 · similarity 0.84)
- Neural-Symbolic Recursive Machine for Systematic Generalization (ICLR 2024 · similarity 0.84)
- ORACLE: Optimizing Reasoning Abilities of Large Language Models via Constraint-Led Synthetic Data Elicitation (AAAI 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.