Research map / Continual learning / Continual fine-tuning of LLMs
Just-In-Time Reinforcement Learning: Continual Learning in LLM Agents Without Gradient Updates
Yibo Li, Zijie Lin, Ailin Deng, Xuan Zhang, Yufei He, Shuo Ji, Tri Cao, Bryan Hooi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| update output logits with memory-based advantages instead of appending them to the prompt (ablation) | JitRL · WebArena Admin | success rate (%): 48.35 -> 52.31 | Improved |
| update output logits with memory-based advantages instead of appending them to the prompt (ablation) | JitRL · WebArena Reddit | success rate (%): 54.57 -> 57.64 | Improved |
| use JitRL instead of static agent | JitRL · WebArena average | success rate (%): 35.63 -> 46.98 | Improved |
| use JitRL instead of WebRL fine-tuning on held-out WebArena-Lite | JitRL · WebArena-Lite | final success rate (%): 46.06 -> 60.00 | Improved |
| use JitRL instead of GRPO training on Zork1 | JitRL · Jericho Zork1 | average score: 16.2 -> 53.0 | Improved |
| add Reflexion self-reflection on Map site | Reflexion · WebArena Map | average success rate (%): 30.62 -> 29.38 | Hurt |
| use JitRL with DeepSeek-V3.2 backbone instead of Static | JitRL · WebArena Admin (DeepSeek-V3.2) | average success rate (%): 32.07 -> 50.65 | Improved |
| use JitRL with GPT-5-mini backbone instead of Static | JitRL · WebArena Admin (GPT-5-mini) | average success rate (%): 37.07 -> 48.04 | Improved |
| restrict memory retrieval to disjoint tasks (unseen-task generalization) | JitRL · WebArena Shopping | average success rate (%): 23.44 -> 36.98 | Improved |
| use JitRL instead of Reflexion on Reddit with DeepSeek-V3.2 | JitRL · WebArena Reddit (DeepSeek-V3.2) | average success rate (%): 55.04 -> 54.42 | Hurt |
| set retrieval neighbor count k within 8-14 instead of outside (ablation) | JitRL · Library and Zork1 | score | Improved |
| use JitRL instead of WebRL in training cost | JitRL · WebArena | cost (USD): ~$9900 -> $290 | Improved |
Closest papers by meaning
- From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memory (ICML 2026 · similarity 0.89)
- ShiQ: Bringing back Bellman to LLMs (NeurIPS 2025 · similarity 0.88)
- Exploratory Memory-Augmented LLM Agent via Hybrid On- and Off-Policy Optimization (ICLR 2026 · similarity 0.88)
- LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models (NeurIPS 2025 · similarity 0.88)
- VESPO: Variational Sequence-level Soft Policy Optimization for Off-Policy LLM Training (NeurIPS 2026 · similarity 0.88)
- Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
