Research map / Video / Video segmentation and tracking
Predicting Deeper Into the Future of Semantic Segmentation
Pauline Luc, Natalia Neverova, Camille Couprie, Jakob Verbeek, Yann LeCun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| predict segmentations directly instead of predicting RGB then segmenting | S->S vs X->X · Cityscapes short-term | IoU GT: 23.0 -> 58.3 | Improved |
| add RGB input to segmentation input (ablation) | S->S · Cityscapes short-term | IoU GT: 58.3 -> 58.2 | No clear effect |
| also predict RGB frames jointly (ablation) | S->S · Cityscapes short-term | IoU GT: 58.3 -> 55.5 | Hurt |
| add adversarial fine-tuning (ablation) | S->S · Cityscapes short-term | IoU GT: 58.3 -> 58.3 | No clear effect |
| use dilated-convolution architecture | S->S · Cityscapes short-term | IoU GT: 58.3 -> 59.4 | Improved |
| use two scales instead of one (ablation) | S->S · Cityscapes short-term | IoU GT: 57.7 -> 58.3 | Improved |
| add gradient difference loss to l1 (ablation) | S->S · Cityscapes short-term | IoU GT: 57.6 -> 58.3 | Improved |
| train on softmax pre-activations with l1+gdl instead of MCE on labels (ablation) | S->S · Cityscapes short-term | IoU GT: 55.5 -> 58.3 | Improved |
| autoregressive instead of batch prediction (ablation) | S->S · Cityscapes mid-term | IoU GT: 42.1 -> 45.3 | Improved |
| autoregressive fine-tuning (ablation) | S->S AR · Cityscapes mid-term | IoU GT: 45.3 -> 46.7 | Improved |
| autoregressive instead of batch prediction (ablation) | X->X (RGB) · Cityscapes frame 20 | PSNR: 20.6 -> 19.2 | Hurt |
| add adversarial fine-tuning (ablation) | S->S AR · Cityscapes mid-term | IoU GT: 45.3 -> 45.1 | No clear effect |
Closest papers by meaning
- Predicting Future Instance Segmentation by Forecasting Convolutional Features (ECCV 2018 · similarity 0.92)
- Probabilistic Future Prediction for Video Scene Understanding (ECCV 2020 · similarity 0.90)
- Predicting Scene Parsing and Motion Dynamics in the Future (NeurIPS 2017 · similarity 0.89)
- Representations Before Pixels: Semantics-Guided Hierarchical Video Prediction (ECCV 2026 · similarity 0.88)
- An Uncertain Future: Forecasting from Static Images Using Variational Autoencoders (ECCV 2016 · similarity 0.88)
- Advancing Semantic Future Prediction through Multimodal Visual Sequence Transformers (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
