Research map / Continual learning / Domain and prompt-based
Not Just Object, But State: Compositional Incremental Learning without Forgetting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces composition-IL, a task for incrementally learning state-object compositions, and CompILer, a prompt-based rehearsal-free learner with multi-pool prompts, object-injected state prompting, and generalized-mean fusion, evaluated on two re-organized datasets.
Problem
Incremental learners prioritize object classes and neglect attached states, so state-object compositions become ambiguous when new compositions arrive over tasks while primitives recur.
Approach
Uses three prompt pools (state, object, composition) regularized by inter-pool discrepancy and intra-pool diversity losses, injects object prompts into the query for state prompt selection, and fuses selected prompts with a learnable generalized-mean operator on a frozen ViT.
Key results
- CompILer reaches 89.21% Avg Acc on Split-Clothing (5 tasks) vs 87.87% for Dual-Prompt and 80.22% for L2P.
- On Split-UT-Zappos (5 tasks) CompILer achieves 46.48% Avg Acc vs 43.35% for CODA-Prompt; on 10 tasks it gets 34.43% vs 33.56% for LGCL.
- Ablation: object-injected state prompting (O→S) improves Split-Clothing Avg Acc from 88.45 to 89.21 and HM from 93.70 to 94.18.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CompILer instead of the strongest baseline | Incremental composition recognition · Split-Clothing (5 tasks) | Avg Acc: 87.87±0.63 -> 89.21±0.24 | Helped |
| Use CompILer instead of the strongest baseline | Incremental composition recognition · Split-UT-Zappos (5 tasks) | Avg Acc: 43.35±0.29 -> 46.48±0.26 | Helped |
| Use CompILer instead of the strongest baseline | Incremental composition recognition · Split-UT-Zappos (10 tasks) | Avg Acc: 33.56±0.31 -> 34.43±0.07 | Helped |
| Add object-injected state prompting (ablation) | CompILer · Split-Clothing (5 tasks) | Avg Acc: 88.45±0.10 -> 89.21±0.24 | Helped |
| Use state-injected object prompting instead of no injected prompt selection (ablation) | CompILer · Split-Clothing (5 tasks) | Avg Acc: 88.45±0.10 -> 88.27±0.02 | Hurt |
| Use GeM rather than max prompt fusion (ablation) | CompILer · Split-Clothing (5 tasks) | Avg Acc: 84.70±0.64 -> 89.21±0.24 | Helped |
| Use GeM rather than mean prompt fusion (ablation) | CompILer · Split-Clothing (5 tasks) | Avg Acc: 87.80±0.12 -> 89.21±0.24 | Helped |
| Add an object prompt pool to the composition-only prompt pool (ablation) | CompILer · Split-Clothing (5 tasks) | Avg Acc: 80.22±0.41 -> 88.10±0.11 | Helped |
| Add a state prompt pool to the composition-only prompt pool (ablation) | CompILer · Split-Clothing (5 tasks) | Avg Acc: 80.22±0.41 -> 88.09±0.50 | Helped |
| Add both state and object prompt pools (ablation) | CompILer · Split-Clothing (5 tasks) | Avg Acc: 80.22±0.41 -> 88.38±0.08 | Helped |
| Add reverse cross-entropy loss (ablation) | CompILer · Split-Clothing (5 tasks) | Avg Acc: 88.17±0.08 -> 88.36±0.37 | Helped |
| Add inter-pool discrepancy loss (ablation) | CompILer · Split-Clothing (5 tasks) | Avg Acc: 88.17±0.08 -> 88.32±0.56 | Helped |
Surprising findings
- Applying S→O (state-injected object prompting) decreases all metrics, suggesting state prompts interfere with object prompt selection.
- Prompt-free methods achieve higher state than object accuracy on Split-Clothing because color-related states are easier to capture by fine-tuning.
Limitations
- LGCL cannot run on 5-task Split-UT-Zappos due to class-name length limits of its CLIP text guidance.
- CompILer falls behind Dual-Prompt and LGCL on forgetting (FTT) for the 10-task Split-UT-Zappos setting.
- Datasets are re-organized subsets (35 and 80 compositions), so scale is limited.
Experimental setup
Datasets: Clothing16K UT-Zappos50K Split-Clothing Split-UT-Zappos
Benchmarks: Split-Clothing (5 tasks) Split-UT-Zappos (5 tasks) Split-UT-Zappos (10 tasks)
Metrics: Avg Acc FTT (forgetting) State accuracy Object accuracy HM
Baselines: EWC LwF iCaRL L2P Deep L2P++ Dual-Prompt CODA-Prompt LGCL Sim-CompILer
Builds on: L2P Dual-Prompt CODA-Prompt symmetric cross entropy directional decoupled loss
Closest papers by meaning
- Learning to Compose: Improving Object Centric Learning by Injecting Compositionality (ICLR 2024 · similarity 0.88)
- Compositional Few-Shot Class-Incremental Learning (ICML 2024 · similarity 0.87)
- A Second-Order Perspective on Model Compositionality and Incremental Learning (ICLR 2025 · similarity 0.86)
- Unlocking Compositional Generalization in Continual Few-Shot Learning (NeurIPS 2026 · similarity 0.86)
- Prompt Tuning In a Compact Attribute Space (AAAI 2025 · similarity 0.85)
- SEC-Prompt:SEmantic Complementary Prompting for Few-Shot Class-Incremental Learning (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.