Research map / Continual learning / Continual fine-tuning of LLMs
ECA: Efficient Continual Alignment for Open-Ended Image-to-Text Generation
Jiangtao Kong, Peijun Zhao, Chun-Fu Chen, Youngwook Do, Shaohan Hu, Tianyi Zhou, Huajie Shao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Mixture of Query (shared orthogonal task queries) to a parallel adapter (ablation) | Vanilla (PA) on BLIP-2 · ToS-COCO Caption | CIDEr (Avg): 123.00 -> 123.67 | Improved |
| use per-task query tokens without cross-task sharing instead of MoQ (ablation) | Vanilla (PA) on BLIP-2 · ToS-COCO Caption | CIDEr (Avg): 123.00 -> 122.74 | No clear effect |
| add Dictionary Replay on top of MoQ (ablation) | PA+MoQ on BLIP-2 · ToS-COCO Caption | CIDEr (Avg): 123.67 -> 124.57 | Improved |
| replay a randomly initialized dictionary instead of a learned one (ablation) | PA+MoQ+DR on BLIP-2 · ToS-COCO Caption | CIDEr (Avg): 124.57 -> 123.24 | Hurt |
| add Fisher Dynamic Expansion on top of MoQ (ablation) | PA+MoQ on BLIP-2 · ToS-COCO Caption | CIDEr (Avg): 123.67 -> 124.95 | Improved |
| add Dictionary Replay to MoQ+FeDEx (full ECA) (ablation) | PA+MoQ+FeDEx on BLIP-2 · ToS-COCO Caption | CIDEr (Avg): 124.95 -> 125.56 | Improved |
| use orthogonality loss only in MoQ (ablation) | PA+MoQ on BLIP-2 · ToS-COCO Caption | CIDEr (BWT): -4.50 -> -3.99 | Improved |
| use key-alignment loss only in MoQ (no orthogonality) (ablation) | PA+MoQ on BLIP-2 · ToS-COCO Caption | BLEU-4 (BWT): -1.49 -> -1.80 | Hurt |
| set DR weight lambda to 1 instead of 0.1 (ablation) | PA+MoQ+DR · ToS-COCO Caption | CIDEr (Avg): 124.57 -> 122.84 | Hurt |
| use 2.5x dv dictionary atoms instead of 5x (ablation) | PA+MoQ+DR · ToS-COCO Caption | CIDEr (Avg): 124.57 -> 122.42 | Hurt |
| set FeDEx expansion threshold S below 0.5 (ablation) | ECA · ToS-COCO Caption | Avg metrics | Hurt |
| add EWC regularization to a single parallel adapter | Vanilla (PA) on BLIP-2 · ToS-TextCaps | CIDEr (Avg): 89.39 -> 86.98 | Hurt |
Closest papers by meaning
- Cycle Consistency as Reward: Learning Image-Text Alignment without Human Preferences (ICCV 2025 · similarity 0.87)
- Aligning Text to Image in Diffusion Models is Easier Than You Think (NeurIPS 2025 · similarity 0.87)
- Visual-Text Cross Alignment: Refining the Similarity Score in Vision-Language Models (ICML 2024 · similarity 0.87)
- SILMM: Self-Improving Large Multimodal Models for Compositional Text-to-Image Generation (CVPR 2025 · similarity 0.86)
- Improving Long-Text Alignment for Text-to-Image Diffusion Models (ICLR 2025 · similarity 0.86)
- Text as Partial Constraint: Core–Residual Alignment for Robust Vision–Language Learning (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
