Research map / Continual learning / Continual fine-tuning of LLMs

Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning

NeurIPS 2026 · Continual learning › Continual fine-tuning of LLMs · approach: Continual pretraining of language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RaPO, an RFT (GRPO-based) method for rehearsal-free visual continual learning that adds a drift-based retention reward and cross-task EMA advantage normalization to reduce forgetting.

Problem

Whether reinforcement fine-tuning with verifiable rewards can overcome catastrophic forgetting in challenging rehearsal-free visual continual learning (class-incremental and domain-incremental), where GRPO still forgets significantly.

Approach

Converts the per-rollout token-averaged log-ratio drift from the preceding-task policy into a bounded exponential retention reward added to the task reward before group-relative advantage computation, plus a persistent EMA reward-scale normalizer (CTAN) carried across task boundaries.

Key results

• On ImageNet-R 10-task CIL, RaPO reaches 85.92% last accuracy and 4.69% forgetting versus 74.67% / 20.02% for GRPO.
• On COCO 5-task class-incremental detection, RaPO gets 19.31 box AP and 1.39 box forgetting versus 14.64 / 6.67 for GRPO.
• Ablation: the retention reward alone raises ImageNet-R accuracy from 74.67 to 82.57, and CTAN alone to 79.43.

Surprising findings

• A standard loss-level KL term has only a modest effect on continual performance (RaPO without KL loss still gets 84.38 on ImageNet-R).
• LwF becomes competitive with GRPO in class-incremental object detection, unlike in image classification.

Limitations

• Retention reward is activated only from task 2, and anchor is the previous-task policy, which requires storing a frozen model copy.
• Theoretical convergence analysis is idealized and does not give a formal no-forgetting guarantee, as the authors state.
• Experiments use few-shot (5 per class) settings and a relatively small set of MLLMs (Qwen2-VL 2B and 7B).

Experimental setup

Datasets: ImageNet-R ImageNet-A TinyImageNet CUB-200 COCO 2017 UCF-101 Kinetics-200 DomainNet OfficeHome Pascal VOC Clipart Watercolor

Benchmarks: ImageNet-R ImageNet-A TinyImageNet CUB-200 COCO 2017 UCF-101 Kinetics-200 DomainNet OfficeHome Pascal Series

Metrics: last accuracy forgetting box AP box forgetting

Baselines: SFT GRPO L2 EWC LwF

Builds on: GRPO EWC LwF on-policy distillation

Closest papers by meaning

• Stabilizing Zero-Shot Prediction: A Novel Antidote to Forgetting in Continual Vision-Language Tasks (NeurIPS 2024 · similarity 0.89)
• Synthetic Data is an Elegant GIFT for Continual Vision-Language Models (CVPR 2025 · similarity 0.89)
• Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting (ICML 2025 · similarity 0.89)
• STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.89)
• Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective (ICML 2025 · similarity 0.89)
• Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective (ICML 2025 · similarity 0.88)