Research map / Large language models / Language, safety and interpretability

Cross-Dialect Generalization Without Retraining: Benchmarks and Evaluation of Schema-Derived Constrained Decoding for MLIR

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Parallel and early-exit decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Releases four NL-to-MLIR benchmarks across three dialects and shows that a training-free, ODS-derived three-layer constrained decoding stack lets SmolLM2-1.7B match or beat 15B-34B code LMs on linalg verify-valid rate, while not winning on arith+func.

Problem

Small language models rarely produce valid MLIR from natural language because MLIR is scarce in code pretraining corpora, and maintaining a fine-tuned model per extensible MLIR dialect does not scale.

Approach

Derive a three-layer constraint stack mechanically from each dialect's ODS TableGen records (CFG over op signatures, type-domain splits from a type lattice, and an SSA-scope validator with five-retry rejection sampling) and apply it at inference time to a small LM, with no fine-tuning, so the same derivation ports across dialects.</edit_text>

Key results

• On linalg, SmolLM2-1.7B with C1+C2+C3 reaches 80.0% verify-valid (three-seed mean, n=125), beating CodeLlama-34B (58.7%), Granite-34B (35.7%) and StarCoder2-15B (54.9%) with non-overlapping CIs.
• On arith+func, SmolLM2 (53.2%) is within CI of CodeLlama-34B (59.8%) and Granite-34B (51.5%) and trails StarCoder2-15B (66.8%), so it is reported as a non-win.
• On StableHLO-Spec-30, SmolLM2 reaches 63.3% vs 33.3-60.0% for baselines, but on Held-Out-200 the baselines saturate at 98-100% while SmolLM2 gets 61.5%.

Surprising findings

• On CodeLlama-34B, adding C3 drops verify-pass by 22.2 pp on arith+func, indicating the SSA-scope validator can false-reject larger-model outputs.
• The in-line coupled decoder reached parse-valid on only 20% of prompts, so the reported results all use post-hoc rejection sampling.

Limitations

• Baselines at 15B/34B use rejection sampling over a LARK grammar via Ollama rather than token-level masking, an asymmetric comparison.
• Verify-valid is structural; functional output-match on the n=30 reference set is low (20% linalg), so structural validity overstates capability.
• Single SLM family and single laptop hardware; StableHLO win is confined to the hand-curated Spec-30 corpus.

Experimental setup

Datasets: MLIR test corpus (filtered subsamples as prompt fillers) ODS TableGen records from LLVM, IREE, StableHLO

Benchmarks: MLIR-Spec-150 Linalg-Spec-30 StableHLO-Spec-30 StableHLO-Held-Out-200 StableHLO-Out-Of-Grammar-25 MLIR-Functional-Reference-30

Metrics: verify-valid rate output-match rate parse-valid rate seconds per generation

Baselines: CodeLlama-34B Granite-Code-34B StarCoder2-15B Granite-Code-8B-fp16

Builds on: Outlines Synchromesh Grammar-Aligned Decoding Type-Constrained Code Generation

Closest papers by meaning

• InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct (AAAI 2025 · similarity 0.87)
• CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidance (ICML 2025 · similarity 0.86)
• Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA (ICLR 2025 · similarity 0.86)
• Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads (ICML 2024 · similarity 0.86)
• Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models (NeurIPS 2024 · similarity 0.86)
• AlchemistCoder: Harmonizing and Eliciting Code Capability by Hindsight Tuning on Multi-source Data (NeurIPS 2024 · similarity 0.85)