Research map / Large language models / Agents, code and math

Program-as-Weights: A Programming Paradigm for Fuzzy Functions

NeurIPS 2026 · Large language models › Agents, code and math · approach: LLM-guided program synthesis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Program-as-Weights, which compiles natural-language specs of fuzzy functions into LoRA adapters via a 4B trained compiler and runs them on a frozen 0.6B Qwen3 interpreter, trained on the released 10M-example FuzzyBench.

Problem

Many everyday programming tasks (log alerting, JSON repair, intent-based ranking) resist rule-based implementation and are outsourced to LLM APIs, which is costly, fragile, irreproducible and not self-contained.

Approach

A neural compiler maps a natural-language spec to a hybrid program (discrete pseudo-program plus a per-example LoRA generated from the compiler's hidden states via a shared-basis mapper), which is executed by a frozen small interpreter, so each new fuzzy function needs only a compile step rather than per-input LLM calls.

Key results

• A 0.6B Qwen3 interpreter running PAW programs reaches 73.78% exact match on FuzzyBench, above prompting Qwen3-32B (68.70%) at roughly 50x less inference memory.
• Quantized PAW runs at about 31.6 tokens/s on a MacBook M3 with a ~484 MB base and 23 MB per-program adapter.
• PAW exceeds full fine-tuning of the same 0.6B base by 15.4 points (73.78% vs 58.40%).

Surprising findings

• A GPT-2 124M interpreter reaches 54% on FuzzyBench using compiler-generated LoRAs despite being far smaller and not instruction-tuned.
• More expressive LoRA mapper variants (per-position aggregation, per-layer bases) underperformed the simple shared-basis default.

Limitations

• Long-form structured generation (Im2LaTeX) is weaker for the LoRA variant than the prefix-tuning precursor, attributed to pseudo-program examples crowding the context budget.
• Compiler scaling results are inconclusive and non-monotonic.
• Test labels are model-generated, and the verified test set depends on agreement between two LLMs, with the empirical ceiling set by gpt-5.2 at 96.09%.

Experimental setup

Datasets: FuzzyBench-10M YouTube spam SMS spam Yelp IMDB CoSyn-400K Im2LaTeX-100K Im2SMILES-20K TextVQA

Benchmarks: FuzzyBench YouTube SMS Yelp IMDB CoSyn Circuit CoSyn Chemical CoSyn Music Im2SMILES Im2LaTeX TextVQA ToolCall-15

Metrics: exact match accuracy F1 tokens/s

Baselines: Qwen3 0.6B Qwen3 4B Qwen3 8B Qwen3 14B Qwen3 32B OLMo3 7B gpt-oss-20B gpt-5-mini gpt-5.2 ALCHEmist LM-to-code full fine-tuning fixed LoRA

Builds on: Text-to-LoRA prefix-tuning LoRA ALCHEmist

Closest papers by meaning

• Gradient-Based Program Synthesis with Neurally Interpreted Languages (ICLR 2026 · similarity 0.85)
• BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions (ICLR 2025 · similarity 0.85)
• Latent Execution for Neural Program Synthesis Beyond Domain-Specific Languages (NeurIPS 2021 · similarity 0.85)
• QiMeng-NeuComBack: Self-Evolving Translation from IR to Assembly Code (NeurIPS 2025 · similarity 0.83)
• Once Upon an Input: Reasoning via Per-Instance Program Synthesis (NeurIPS 2025 · similarity 0.83)
• REASONING COMPILER: LLM-Guided Optimizations for Efficient Model Serving (NeurIPS 2025 · similarity 0.83)