Research map / Continual learning / Continual fine-tuning of LLMs

Filling Memory Gaps: Enhancing Continual Semantic Parsing via SQL Syntax Variance-Guided LLMs Without Real Data Replay

AAAI 2025 · Continual learning › Continual fine-tuning of LLMs · approach: Continual pretraining of language models · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LECSP, a continual semantic parsing method that uses SQL-skeleton variance to guide an LLM (Mixtral-8x7B) in generating pseudo-memory samples, then trains T5 students with a task-aware dual-teacher distillation, without real data replay.

Problem

Continual semantic parsing (text-to-SQL over a stream of databases with limited annotated data) suffers from catastrophic forgetting, while existing methods rely on replaying real historical data, PET modules with ideal task-identity assumptions, or extra unlabeled data.

Approach

Analyze SQL-skeleton component bias between current and past tasks (via CodeT5 encoding and K-means) to prompt an LLM to generate intra- and inter-task pseudo-samples, calibrate them via execution-based iterative self-correction and skeleton-based sampling, and distill from both the LLM and the previous student model.

Key results

• On Spider-stream-semi (T5-large, warm start), LECSP reaches 58.6 EM ACC_a without replay, versus 52.2 for SFNET with replay, and up to 8.8% gains over SFNET.
• Forward transfer of LECSP (T5-large) reaches 41.4 EM on Spider-stream-semi, exceeding the ORACLE upper bound of 34.3 on the FWT metric.
• Under cold start, LECSP gains up to 11.4% over SFNET with replay; removing memory calibration drops ACC_a by up to 31.4% (T5-base).

Measured effects reported in the paper

Surprising findings

• Without memory calibration, LECSP's average accuracy collapses to 20.3 EM from 47.4, indicating LLM hallucinated pseudo-samples are very harmful.
• Zero-shot Mixtral-8x7B reaches only 22.0 EM, far below the student trained with its generated memory (47.4 EM).

Limitations

• Relies on a 56B-parameter LLM teacher, with inference cost for pseudo-sample generation and calibration.
• Rehearsal comparison is handicapped since the method uses no real replay, and BWT is comparable to SFNET only on some difficulty levels.
• Pseudo-samples require manual evaluation for quality analysis; hallucination still a risk.

Experimental setup

Datasets: Spider-stream-semi Combined-stream Spider WikiSQL

Benchmarks: Spider-stream-semi Combined-stream

Metrics: EM EX ACC_a ACC_w BWT FWT

Baselines: SFT EMAR SFNET PEFT C3 ProgPrompt ORACLE

Builds on: EMAR SFNET PEFT CodeT5

Closest papers by meaning

• Scalable Language Model with Generalized Continual Learning (ICLR 2024 · similarity 0.86)
• CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models (AAAI 2025 · similarity 0.86)
• K-DeCore: Facilitating Knowledge Transfer in Continual Structured Knowledge Reasoning via Knowledge Decoupling (NeurIPS 2025 · similarity 0.85)
• CogSQL: A Cognitive Framework for Enhancing Large Language Models in Text-to-SQL Translation (AAAI 2025 · similarity 0.85)
• Learn more, but bother less: parameter efficient continual learning (NeurIPS 2024 · similarity 0.85)
• Data Engineering for Scaling Language Models to 128K Context (ICML 2024 · similarity 0.85)