Research map / Large language models / Language, safety and interpretability
Joint Prompt Optimization of Stacked LLMs using Variational Inference
Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, Nicolas Le Roux
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| optimize a single-layer prompt rather than use the strongest DaVinci-003 baseline | DLN-1 · Hyper. test | accuracy: 68.5{ $\pm$5.5} -> 91.9{ $\pm$3.0} | Improved |
| optimize a single-layer prompt rather than use the strongest DaVinci-003 baseline | DLN-1 · Trec test | accuracy: 80.0 -> 89.7{ $\pm$3.2} | Improved |
| optimize a single-layer prompt rather than use the strongest DaVinci-003 baseline | DLN-1 · Subj test | accuracy: 76.4 -> 83.2{ $\pm$6.5} | Improved |
| optimize a single-layer prompt rather than use the strongest DaVinci-003 baseline | DLN-1 · Nav. test | accuracy: 67.3{ $\pm$7.7} -> 68.5{ $\pm$4.7} | Improved |
| optimize a single-layer prompt rather than use the strongest DaVinci-003 baseline | DLN-1 · Logic.7 test | accuracy: 45.5{ $\pm$4.7} -> 47.5{ $\pm$2.1} | Improved |
| optimize a single-layer prompt rather than use the zero-shot baseline | DLN-1 · Date. test | accuracy: 56.4 -> 55.7{ $\pm$4.5} | Hurt |
| add a jointly optimized second LLM layer | DLN-1 · Nav. test | accuracy: 68.5 -> 83.1 | Improved |
| add a jointly optimized second LLM layer | DLN-1 · Date. test | accuracy: 55.7 -> 75.2 | Improved |
| add a jointly optimized second LLM layer | DLN-1 · Logic.7 test | accuracy: 47.5 -> 45.7 | Hurt |
| add a jointly optimized second LLM layer | DLN-1 · Disaster test | accuracy: 81.7 -> 82.8 | Improved |
| add a jointly optimized second LLM layer | DLN-1 · Subj test | accuracy: 83.2 -> 85.9 | Improved |
| train layerwise and keep the second layer fixed instead of end-to-end training (ablation) | DLN-2 · Nav. test | accuracy: 73.1 vs 83.1 | Hurt |
Closest papers by meaning
- Skip a Layer or Loop It? Learning Program-of-Layers in LLMs (ICML 2026 · similarity 0.89)
- Search for Efficient Large Language Models (NeurIPS 2024 · similarity 0.89)
- Benchmarking Optimizers for Large Language Model Pretraining (NeurIPS 2026 · similarity 0.89)
- Concentrate Attention: Towards Domain-Generalizable Prompt Optimization for Language Models (NeurIPS 2024 · similarity 0.88)
- Deep Fusion: Efficient Network Training via Pre-trained Initializations (ICML 2024 · similarity 0.88)
- Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
