Research map / Continual learning / Domain and prompt-based
SimCS: Simulation for Domain Incremental Online Continual Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ODICS, an online domain-incremental continual semantic segmentation benchmark over four driving datasets, and SimCS, a parameter-free regularizer that adds loss on on-the-fly simulated CARLA/VIPER data to existing continual learning methods.
Problem
Continual semantic segmentation under realistic online, domain-incremental conditions (no domain boundaries, limited per-step computation, no replay due to privacy) where existing class-incremental methods perform poorly.
Approach
Use the segmentation loss on photo-realistic simulated driving images generated on the fly as a privacy-safe regularizer that can be added to any CL method (naive, EWC, MAS, LwF, ER), reducing forgetting without storing real data.
Key results
- SimCS with VIPER raises naive training mIOU from 40.5% to 44.9% on the ODICS benchmark, and improves ER from 46.2% to 48.3%.
- SimCS improved every considered CL method (NT, EWC, MAS, LwF, ER) with both CARLA and VIPER across all four domains.
- Pre-training on VIPER plus SimCS during continual learning lifts NT to 46.7% mIOU, versus 40.5% for NT alone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add simulated data from CARLA or VIPER during continual learning | Naive Training, EWC, MAS, LwF, and ER · ODICS (CS-IDD-BDD-ACDC) | mIOU: 40.5 -> 43.4 (CARLA); 40.5 -> 44.9 (VIPER); 46.2 -> 47.7 (CARLA, ER); 46.2 -> 48.3 (VIPER, ER) | Helped |
| Pretrain on VIPER simulated data before continual learning (ablation) | Naive Training · ODICS (CS-IDD-BDD-ACDC) | mIOU: 40.5 -> 42.3 | Helped |
| Add VIPER SimCS during continual learning after VIPER pretraining (ablation) | Naive Training pretrained on VIPER · ODICS (CS-IDD-BDD-ACDC) | mIOU: 42.3 -> 46.7 | Helped |
| Increase the replay buffer size (ablation) | Experience Replay · ODICS (CS-IDD-BDD-ACDC) | mIOU: 45.9 (ER(200)); 46.2 (ER(800)); 47.4 (ER(1000)); 48.3 (ER(1200)) | Helped |
| Increase the simulated-to-real data ratio to 10 (ablation) | Naive Training + SimCS · ODICS real-world domains | mIOU | Hurt |
| Use simulated-to-real ratios smaller than 5 (ablation) | Naive Training + SimCS · ODICS real-world domains | mIOU | Helped |
| Increase the training budget to N=10 (ablation) | Naive Training and SimCS · ODICS, previous domains | mIOU | Hurt |
| Add VIPER SimCS with normalized computation budget (ablation) | Naive Training · IDD | mIOU: 39.1 -> 43.8 | Helped |
| Add VIPER simulated data to supervised training (ablation) | Supervised segmentation · ODICS stream | mIOU: 59.5 -> 59.8 | No clear effect |
| Add VIPER SimCS to improve forward transfer (ablation) | Naive Training · ACDC after training on CS-IDD-BDD | mIOU: 34.9% to 39% | Helped |
| Add simulated data during data-incremental training (ablation) | Naive Training · Data-incremental segmentation | mIOU: 42.1 -> 43.4 | Helped |
| Increase computation from N=1 to N=4 (ablation) | All continual learning methods · ODICS | mIOU | Helped |
Surprising findings
- Regularization-based methods (EWC, MAS, LwF) perform on par with naive training in the domain-incremental online setting, unlike in class-incremental segmentation.
- Simulated data does not help in the fully supervised setting (59.5% to 59.8% mIOU), so gains come from reduced forgetting rather than extra data.
Limitations
- Simulated labels cover only 13 (CARLA) and 15 (VIPER) of 19 classes, with dropped labels excluded from loss.
- Regularization-based methods required domain boundary information, an exception to the boundary-free setting.
- Benchmark covers only four driving datasets in a single label space and a fixed domain order by year.
Experimental setup
Datasets: Cityscapes Indian Driving Dataset Berkeley Driving Dataset ACDC CARLA VIPER
Benchmarks: ODICS benchmark (CS-IDD-BDD-ACDC stream)
Metrics: mIOU forgetting forward transfer
Baselines: Naive Training EWC MAS LwF Experience Replay Supervised upper bound
Builds on: Elastic Weight Consolidation Memory Aware Synapses Learning without Forgetting Experience Replay DeepLabV3
Closest papers by meaning
- Rethinking Query-based Transformer for Continual Image Segmentation (CVPR 2025 · similarity 0.86)
- ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning (CVPR 2024 · similarity 0.86)
- Towards Continual Universal Segmentation (CVPR 2025 · similarity 0.85)
- Continual Segmentation with Disentangled Objectness Learning and Class Recognition (CVPR 2024 · similarity 0.85)
- FR²Seg: Continual Segmentation Across Multiple Sites via Fourier Style Replay and Adaptive Consistency Regularization (AAAI 2025 · similarity 0.84)
- Beyond Background Shift: Rethinking Instance Replay in Continual Semantic Segmentation (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.