Research map / Large language models / Retrieval and knowledge

BLADE: Enhancing Black-Box Large Language Models with Small Domain-Specific Models

AAAI 2025 · Large language models › Retrieval and knowledge · approach: Knowledge-augmented question answering · 11 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BLADE, which pairs a black-box general LLM with a small (<3B) domain-specific LM trained via domain pretraining, knowledge instruction tuning from LLM-generated knowledge, and Bayesian prompt optimization over soft prompts, improving zero-shot legal and medical QA accuracy.

Problem

General LLMs lack domain-specific knowledge for vertical domains like law and medicine, while continued pretraining is costly and requires weight access, and retrieval augmentation is unreliable.

Approach

Treat the general LLM as a black box and train a small domain LM to generate question-specific knowledge that the black-box LLM consumes as context; the small LM is trained with domain pretraining, consistency-filtered knowledge instruction tuning, and derivative-free Bayesian optimization of soft prompts through the black-box LLM's accuracy.

Key results

• BLADE improves ChatGPT on JEC-QA from 19.38% to 25.46% overall accuracy (+31.3% relative), outperforming the best M3E-base retrieval augmentation (+22.9%).
• On MLEC-QA, BLADE raises ChatGLM-6B and most general LLMs substantially, e.g., Baichuan-7B Cli accuracy from 27.80 to 54.86.
• On CaseHOLD, BLADE improves Baichuan2-13B-Chat from 48.69 to 62.55, while ChatGPT gains only 3.5%.

Measured effects reported in the paper

Surprising findings

• Legal-specific fine-tuned LLMs such as LexiLaw underperform general ChatGLM-6B on JEC-QA, and continued tuning can degrade prompt-following.
• ChatGPT gains much less from BLADE on CaseHOLD (3.5%), which the authors attribute to possible legal data in ChatGPT's training.

Limitations

• Experiments only on multiple-choice QA datasets; feasibility for generative tasks is untested.
• Relies on ChatGPT for knowledge generation and on accuracy-based optimization that requires labeled training data.
• Gains on ChatGPT for CaseHOLD are modest, and the approach depends on the general LLM's intrinsic knowledge.

Experimental setup

Datasets: JEC-QA CaseHOLD MLEC-QA MultiLegalPile

Benchmarks: JEC-QA CaseHOLD MLEC-QA

Metrics: accuracy relative gain %

Baselines: ChatGLM-6B ChatGLM2-6B Qwen-7B-Chat Baichuan2-13B-Chat ChatGPT LawyerLLaMA ChatLaw-33B Zhongjing Taiyi M3E-base retrieval augmentation

Builds on: retrieval augmentation CMA-ES BLOOMZ

Closest papers by meaning

• Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.89)
• TAIA: Large Language Models are Out-of-Distribution Data Learners (NeurIPS 2024 · similarity 0.88)
• metabench - A Sparse Benchmark of Reasoning and Knowledge in Large Language Models (ICLR 2025 · similarity 0.88)
• Training with “Paraphrasing the Original Text” Teaches LLM to Better Retrieve in Long-Context Tasks (AAAI 2025 · similarity 0.88)
• Once Read is Enough: Domain-specific Pretraining-free Language Models with Cluster-guided Sparse Experts for Long-tail Domain Knowledge (NeurIPS 2024 · similarity 0.88)
• Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.88)