Research map / Continual learning / Domain and prompt-based
ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ECLIPSE, a distillation-free continual panoptic segmentation method that freezes a Mask2Former model and trains only per-step visual prompt embeddings plus MLP heads, with logit manipulation of the no-object logit at inference.
Problem
Continual panoptic segmentation must learn new classes without forgetting old ones, but existing methods rely on knowledge distillation or pseudo-labeling, which raise training complexity and compute; error propagation and no-object semantic drift also remain problems.
Approach
Freeze all trained Mask2Former parameters after each step and learn a new set of visual prompt embeddings with MLP heads for each step's new classes; at inference, aggregate all prompt sets and replace the no-object logit with a modulated sum of other classes' logits (logit manipulation) to suppress propagated errors and semantic drift.
Key results
- On ADE20K 100-10 panoptic (ResNet-50), ECLIPSE reaches 33.9 PQ overall versus 29.7 for CoMFormer, using 0.60M trainable parameters (about 1.3% of total).
- With Swin-L as frozen backbone, ECLIPSE reaches 40.9 PQ on 100-10 and 42.0 on 100-50 panoptic.
- On ADE20K continual semantic segmentation (100-5, ResNet-101), ECLIPSE achieves 34.2 mIoU overall, the best among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ECLIPSE instead of CoMFormer | Mask2Former R50 · ADE20K panoptic 100-5 | PQ all: 28.2 -> 32.9 | Helped |
| use ECLIPSE instead of CoMFormer | Mask2Former R50 · ADE20K panoptic 100-10 | PQ all: 29.7 -> 33.9 | Helped |
| use ECLIPSE instead of CoMFormer | Mask2Former R50 · ADE20K panoptic 100-50 | PQ all: 34.6 -> 35.6 | Helped |
| use ECLIPSE instead of PLOP | Mask2Former R50 · ADE20K panoptic 50-10 | PQ all: 23.3 -> 26.8 | Helped |
| use ECLIPSE instead of CoMFormer | Mask2Former R50 · ADE20K panoptic 50-20 | PQ all: 25.7 -> 28.6 | Helped |
| use ECLIPSE instead of CoMFormer | Mask2Former R50 · ADE20K panoptic 50-50 | PQ all: 27.9 -> 29.2 | Helped |
| use ECLIPSE instead of CoMFormer | Mask2Former R101 · ADE20K semantic 100-5 | mIoU all: 30.9 -> 34.2 | Helped |
| use ECLIPSE instead of PLOP | Mask2Former R101 · ADE20K semantic 100-10 | mIoU all: 28.5 -> 34.6 | Helped |
| use ECLIPSE instead of CoMFormer | Mask2Former R101 · ADE20K semantic 100-50 | mIoU all: 37.6 -> 37.1 | Hurt |
| use 100 prompts instead of 150 (ablation) | ECLIPSE · ADE20K panoptic 100-10 | PQ all: 33.9 -> 33.7 | No clear effect |
| use 200 prompts instead of 150 (ablation) | ECLIPSE · ADE20K panoptic 100-10 | PQ all: 33.9 -> 33.4 | Hurt |
| use 300 prompts instead of 150 (ablation) | ECLIPSE · ADE20K panoptic 100-10 | PQ all: 33.9 -> 32.5 | Hurt |
Surprising findings
- Skipping model freezing and fine-tuning all parameters without distillation collapses performance, showing freezing alone is central to the method.
- Increasing the number of prompts beyond the per-class default does not help and reduces performance due to over-weighting new classes and more false positives.
Limitations
- Computational cost grows with the number of prompt sets as classes increase.
- Mask2Former-based methods achieve slightly higher new-class mIoU than ECLIPSE in the 100-50 semantic scenario.
- Evaluated only on ADE20K.
Experimental setup
Datasets: ADE20K
Benchmarks: ADE20K continual panoptic segmentation ADE20K continual semantic segmentation
Metrics: PQ mIoU
Baselines: FT MiB PLOP CoMFormer SDR UCD SPPA RCIL SSUL REMINDER
Builds on: Mask2Former Visual Prompt Tuning CoMFormer MiB PLOP
Closest papers by meaning
- Towards Continual Universal Segmentation (CVPR 2025 · similarity 0.87)
- Beyond Background Shift: Rethinking Instance Replay in Continual Semantic Segmentation (CVPR 2025 · similarity 0.86)
- SimCS: Simulation for Domain Incremental Online Continual Segmentation (AAAI 2024 · similarity 0.86)
- Rethinking Query-based Transformer for Continual Image Segmentation (CVPR 2025 · similarity 0.85)
- Continual Segmentation with Disentangled Objectness Learning and Class Recognition (CVPR 2024 · similarity 0.85)
- TIKP: Text-to-Image Knowledge Preservation for Continual Semantic Segmentation (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.