Research map / Large language models / Agents, code and math

LILO: Learning Interpretable Libraries by Compressing and Documenting Code

ICLR 2024 · Large language models › Agents, code and math · approach: LLM-guided program synthesis · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

LILO combines LLM-guided program synthesis, Stitch-based compression, and LLM auto-documentation to learn interpretable program libraries that outperform DreamCoder and an LLM-only baseline on REGEX, CLEVR, and LOGO.

Problem

Library learning for program synthesis is computationally expensive with enumerative search (DreamCoder needs CPU-months), and learned libraries are hard to interpret; LLMs struggle to use anonymous abstractions.

Approach

Alternate LLM-guided search with enumerative search, compress solutions with Stitch's branch-and-bound lambda-abstraction refactoring each iteration, and use an LLM to generate human-readable names and docstrings (AutoDoc) for abstractions, which improves both interpretability and downstream LLM synthesis.

Key results

• LILO achieves 77.07% mean test solve rate on REGEX (vs 43.93% DreamCoder) and 48.95% on LOGO (vs 28.53% DreamCoder) in online synthesis.
• On CLEVR, LILO reaches 96.76% mean vs 94.50% for DreamCoder.
• Without AutoDoc, adding Stitch abstractions to the LLM solver hurt performance (e.g., -30.60 points on REGEX relative to LLM Solver), and AutoDoc recovers +9.73 on REGEX.

Measured effects reported in the paper

Surprising findings

• Providing the LLM with raw Stitch abstractions without documentation degraded synthesis performance substantially (-30.60 points on REGEX), which motivated AutoDoc.
• The LLM-only baseline without library learning bootstraps its performance over time, suggesting in-context accumulation acts as non-compressive library learning.

Limitations

• AutoDoc generations sometimes contain ambiguous or incorrect semantic descriptions that may hurt downstream synthesis.
• Experiments are limited to three inductive synthesis domains with a lambda-calculus DSL rather than modern imperative languages.
• Relies on closed-source OpenAI models, limiting reproducibility.

Experimental setup

Datasets: REGEX string editing dataset (LAPS) CLEVR-based scene reasoning dataset (LAPS) LOGO turtle graphics dataset (LAPS)

Benchmarks: REGEX CLEVR LOGO

Metrics: percent of test tasks solved (max, mean, std over 3 seeds)

Baselines: DreamCoder LLM Solver (no library learning) LLM Solver + enumerative search LILO without search LILO without AutoDoc

Builds on: DreamCoder Stitch LAPS

Closest papers by meaning

• HYSYNTH: Context-Free LLM Approximation for Guiding Program Synthesis (NeurIPS 2024 · similarity 0.86)
• BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions (ICLR 2025 · similarity 0.86)
• CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules (ICLR 2024 · similarity 0.86)
• ReGAL: Refactoring Programs to Discover Generalizable Abstractions (ICML 2024 · similarity 0.86)
• EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning (ICML 2025 · similarity 0.86)
• Guess & Sketch: Language Model Guided Transpilation (ICLR 2024 · similarity 0.86)