Research map / Large language models / Agents, code and math
Timely Clinical Diagnosis through Active Test Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ACTMED, which combines Bayesian Experimental Design with LLM-generated surrogate samples of test outcomes to adaptively select clinical tests and stop testing early, evaluated on CKD, hepatitis C, diabetes and a custom OSCE-style set.
Problem
Clinical diagnosis with ML usually relies on static fully observed datasets and ignores the sequential, cost-aware test selection clinicians perform; diagnosis must be made timely under resource limits.
Approach
Use LLMs as zero-shot generative surrogates to sample plausible hidden test outcomes and posterior disease probabilities, then pick the test maximizing expected KL divergence between posterior and prior belief (cost-normalized), with a KL-based stopping rule, without task-specific training.
Key results
- On Diabetes and Hepatitis, ACTMED with GPT-4o outperforms baselines and even the full-information LLM classifier at 3 acquired tests (e.g., hepatitis accuracy 0.839 with KL vs 0.770 entropy-based).
- KL-based stopping at gamma=0.7 reduces average tests to about 1.0-2.3 across datasets, roughly halving diagnostic burden with comparable accuracy.
- Three clinicians and two medical students rated 94.5 ± 1.4% of test selections and risk adjustments as clinically reasonable.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace KL-based selection with entropy-based selection (ablation) | ACTMED · Diabetes, Kidney, and Hepatitis | accuracy: Diabetes: 0.572 ± 0.005 -> 0.593 ± 0.039; Kidney: 0.972 ± 0.010 -> 0.992 ± 0.006; Hepatitis: 0.770 ± 0.010 -> 0.825 ± 0. | Helped |
| Increase stopping threshold γ from 0.3 to 0.7 (ablation) | ACTMED · Hepatitis, Diabetes, and Kidney | average tests selected: GPT-4o-mini: Hepatitis 2.36 ± 0.68 -> 1.48 ± 0.73; Diabetes 2.45 ± 0.71 -> 1.90 ± 0.74; Kidney 1.60 ± 0.75 -> 1.28 ± 0.5 | Helped |
| Use GPT-4o rather than GPT-4o-mini for surrogate sampling | LLM test-outcome surrogate · Diabetes and Hepatitis | Wasserstein distance: Diabetes: 0.117 ± 0.046 -> 0.110 ± 0.038; Hepatitis: 0.173 ± 0.237 -> 0.130 ± 0.157 | Helped |
| Use GPT-4o rather than GPT-4o-mini for surrogate sampling | LLM test-outcome surrogate · Diabetes and Hepatitis | Energy distance: Diabetes: 0.269 ± 0.094 -> 0.256 ± 0.079; Hepatitis: 0.330 ± 0.254 -> 0.265 ± 0.191 | Helped |
| Use ACTMED rather than other feature-selection baselines | Clinical diagnosis · Hepatitis C and Diabetes | diagnostic accuracy | Helped |
| Use ACTMED rather than other feature-selection baselines | Clinical diagnosis · CKD | diagnostic accuracy | No clear effect |
| Apply KL-based early stopping rather than acquiring all three selected tests | ACTMED · Hepatitis, Diabetes, and Kidney | diagnostic accuracy | Helped |
| Use KL-based early stopping rather than baseline feature selection | ACTMED · Hepatitis, Diabetes, and Kidney | number of tests: nearly 50% | Helped |
| Use ACTMED rather than implicit feature selection | Clinical diagnosis · Diabetes | personalization of test selection | Helped |
| Use ACTMED for clinician-reviewed test decisions and risk adjustments | ACTMED · Clinician-in-the-loop evaluation | clinically reasonable judgments: 94.5 ± 1.4% | Helped |
Surprising findings
- Constrained selection of three tests surpassed the full-information LLM baseline on hepatitis and diabetes, suggesting feature selection reduces noise.
- Implicit LLM selection strongly favors globally optimal features, showing limited personalization.
Limitations
- Limited dataset scale (about 1,000 patients total) and binary single-disease framing.
- Relies on strong high-capacity LLMs; smaller models fail to produce accurate surrogate samples.
- Only structured categorical and numerical features; free-text reports not incorporated.
Experimental setup
Datasets: UCI Chronic Kidney Disease UCI HCV data (Hepatitis C) Pima Indians Diabetes AgentClinic (OSCE-style cases)
Benchmarks: UCI Chronic Kidney Disease UCI HCV data (Hepatitis C) Pima Indians Diabetes OSCE-style AgentClinic dataset
Metrics: accuracy Wasserstein distance energy distance mean absolute error of sampled features average number of tests
Baselines: LLM classifier with all features Random selection Global best fixed subset Implicit LLM selection Entropy-based EIG
Builds on: Bayesian Experimental Design active feature acquisition AgentClinic
Closest papers by meaning
- MedExAgent: Training LLM Agents to Ask, Examine, and Diagnose in Noisy Clinical Environments (NeurIPS 2026 · similarity 0.86)
- Large Language Models Are Clinical Reasoners: Reasoning-Aware Diagnosis Framework with Prompt-Generated Rationales (AAAI 2024 · similarity 0.85)
- Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models (NeurIPS 2025 · similarity 0.85)
- CMR-RD: Long-Tailed Adaptive VLM for Explainable CMR Diagnosis (CVPR 2026 · similarity 0.84)
- Aligning Evaluation with Clinical Priorities: Calibration, Label Shift, and Error Costs (NeurIPS 2025 · similarity 0.83)
- ClinicalLab: Aligning Agents for Multi-Departmental Clinical Diagnostics in the Real World (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.