Research map / Continual learning / Continual fine-tuning of LLMs
TRACER: Persistent Regularization for Robust Multimodal Finetuning
Hesam Asadollahzadeh, Feng Liu, Christopher Leckie, Sarah Monazam Erfani
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace static SD anchor with WMA teacher trajectory regularization (ablation) | Static SD · ImageNet distribution shifts (ViT-B/16) | avg shifts accuracy: 59.55 -> 65.46 | Improved |
| add static self-distillation to direct finetuning | Direct FT · ImageNet distribution shifts (ViT-B/16) | avg shifts accuracy: 57.08 -> 59.55 | Improved |
| finetune CLIP directly on ImageNet | CLIP ViT-B/16 zero-shot · ImageNet distribution shifts (ViT-B/16) | avg shifts accuracy: 59.46 -> 57.08 | Hurt |
| add CRD loss alone to contrastive finetuning (ablation) | TRACER base · ImageNet 5-shift avg | OOD accuracy: 59.40 -> 63.12 | Improved |
| add FD loss alone to contrastive finetuning (ablation) | TRACER base · ImageNet 5-shift avg | OOD accuracy: 59.40 -> 63.52 | Improved |
| add ICL loss alone to contrastive finetuning (ablation) | TRACER base · ImageNet 5-shift avg | OOD accuracy: 59.40 -> 59.95 | No clear effect |
| add CrossKD and ICL on top of FD+CRD (ablation) | TRACER · ImageNet 5-shift avg | OOD accuracy: 64.08 -> 64.06 | No clear effect |
| raise distillation strength lambda_SD from 0.1 to 1.5 (ablation) | TRACER · ImageNet 5-shift avg | OOD accuracy: 61.86 -> 64.12 | Improved |
| raise distillation strength lambda_SD from 1.5 to 10.0 (ablation) | TRACER · ImageNet 5-shift avg | OOD accuracy: 64.12 -> 63.27 | Hurt |
| raise distillation strength lambda_SD from 0.1 to 5.0 (ablation) | TRACER · ImageNet 5-shift avg | OOD ECE: 0.1356 -> 0.0670 | Improved |
| update WMA teacher every 100 steps instead of every step (ablation) | TRACER · ImageNet 5-shift avg | OOD accuracy: 64.07 -> 64.22 | No clear effect |
| use dynamic WMA distillation instead of direct finetuning | Direct FT · Colored MNIST toy | forgetting rate on MNIST: 37.9% -> 0.1% | Improved |
Closest papers by meaning
- A Practitioner's Guide to Real-World Continual Multimodal Pretraining (NeurIPS 2024 · similarity 0.87)
- Balancing Multimodal Training Through Game-Theoretic Regularization (NeurIPS 2025 · similarity 0.86)
- Continual Multimodal Contrastive Learning (NeurIPS 2025 · similarity 0.86)
- Synthetic Data is an Elegant GIFT for Continual Vision-Language Models (CVPR 2025 · similarity 0.85)
- A Wander Through the Multimodal Landscape: Efficient Transfer Learning via Low-rank Sequence Multimodal Adapter (AAAI 2025 · similarity 0.85)
- Aligning Multimodal Representations through an Information Bottleneck (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
