Research map / Large language models / Retrieval and knowledge
Continuously Updating Digital Twins using Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Frames digital twinning as in-context learning with LLMs and proposes CALM-DT, which retrieves related trajectories via fine-tuned ModernBERT bi-encoders and adapts context mid-simulation, enabling updates without retraining.
Problem
Digital twins with fixed architectures cannot adapt to new state/action variables, new data, or new knowledge without costly redesign and retraining, which limits their use in dynamic real-world settings.
Approach
Use a frozen LLM as a hybrid digital twin that simulates via in-context learning, with a context-adaptive strategy that re-retrieves knowledge and related trajectories during generation. Retrieval encoders are fine-tuned once using contrastive InfoNCE loss with LLM-simulation performance as labels, and trajectory inputs are augmented with LLM-generated trend summaries.
Key results
- On CF simulation CALM-DT achieves MSE 55.3 and MAE 4.63, the best of all methods (K-NN MSE 65.1, HDTwin 69.8).
- On NSCLC, CALM-DT reaches MSE 79.4 versus HDTwin 80.6 and K-NN 97.3, with the best average rank (1.25) across both datasets.
- Adding a new action with a knowledge-base description and new data reduces post-Ivacaftor MSE from 59.7 to 50.4, and three years of data further reduces it to 47.8, without parameter updates.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CALM-DT instead of the strongest baseline | CF simulation · CF | MSE: 65.1 \pm 0 -> 55.3 \pm 0.811 | Helped |
| use CALM-DT instead of HDTwin | NSCLC simulation · NSCLC | MSE: 80.6 \pm 11.8 -> 79.4\pm8.57 | Helped |
| use CALM-DT instead of HDTwin | NSCLC simulation · NSCLC | MAE: 3.42 \pm 0.717 -> 4.28\pm0.157 | Hurt |
| add encoder fine-tuning and natural-language trajectory summaries (ablation) | CALM-DT sample retrieval · CF | MSE: 76.3 \pm 1.61 -> 55.3 \pm 0.811 | Helped |
| fine-tune the encoders (ablation) | CALM-DT sample retrieval with summaries · CF | MSE: 66.6 \pm 0.387 -> 55.3 \pm 0.811 | Helped |
| append natural-language trajectory summaries (ablation) | CALM-DT sample retrieval with fine-tuned encoders · CF | MSE: 67.1 \pm 0.894 -> 55.3 \pm 0.811 | Helped |
| use GPT-4o instead of GPT-4o Mini as the base LLM (ablation) | CALM-DT · CF | MSE: 64.179 $\pm$ 0.637 -> 55.336 $\pm$ 0.811 | Helped |
| use GPT-4o Mini instead of GPT-3.5 Turbo as the base LLM (ablation) | CALM-DT · CF | MSE: 162.203 $\pm$ 5.367 -> 64.179 $\pm$ 0.637 | Helped |
| increase the context set size from c=0 to c=5 (ablation) | CALM-DT · CF | MSE: 74.816 $\pm$ 0.411 -> 55.336 $\pm$ 0.811 | Helped |
| increase the context set size from c=5 to c=10 (ablation) | CALM-DT · CF | MSE: 55.336 $\pm$ 0.811 -> 59.717 $\pm$ 0.529 | Hurt |
| add a description of the new action's effects to the knowledge base (ablation) | CALM-DT with the new CF action · CF post-Ivacaftor | MSE: 59.7 \pm 0.472 -> 54.4 \pm 0.472 | Helped |
| add post-Ivacaftor patient samples to the dataset (ablation) | CALM-DT with the new CF action and knowledge · CF post-Ivacaftor | MSE: 54.4 \pm 0.472 -> 50.4 \pm 0.477 | Helped |
Surprising findings
- A non-fine-tuned encoder with appended LLM summaries outperforms almost all existing DT baselines, suggesting a training-free LLM-based DT is viable.
- Performance saturates at about 5 context samples; larger context sets (10) degrade MSE.
Limitations
- Inference is slower than other DT methods and closed-source LLM API calls incur monetary cost.
- Hallucinations and output formatting errors can occur, and biases in LLM training corpora may affect performance across populations.
- Relies on closed GPT-family models via Azure; the CF dataset is not publicly accessible.
Experimental setup
Datasets: UK Cystic Fibrosis Registry (2008-2013) NSCLC pharmacokinetic-pharmacodynamic simulator data (500 training patients)
Benchmarks: UK CF registry simulation (CF) NSCLC tumour growth simulation
Metrics: MSE MAE CRPS RMSE
Baselines: 1-NN K-NN Constant RNN Transformer DyNODE SINDy HDTwin
Builds on: HDTwinGen InfoNCE ModernBERT
Closest papers by meaning
- Automatically Learning Hybrid Digital Twins of Dynamical Systems (NeurIPS 2024 · similarity 0.91)
- From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs (NeurIPS 2026 · similarity 0.86)
- TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models (ICLR 2025 · similarity 0.85)
- Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging (NeurIPS 2024 · similarity 0.85)
- A Practitioner's Guide to Real-World Continual Multimodal Pretraining (NeurIPS 2024 · similarity 0.84)
- Dynamics-Predictive Sampling for Active RL Finetuning of Large Reasoning Models (ICLR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.