Research map / Vision-language and multimodal / Human and embodied modeling
Generative Modeling for Multi-task Visual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MGM with weakly labeled synthesized images and joint training instead of MT | Multi-task visual prediction · NYUv2, 50% data | DE-mABSE: 0.819 -> 0.734 | Helped |
| Use MGM with weakly labeled synthesized images and joint training instead of MT | Multi-task visual prediction · NYUv2, 25% data | SS-mIOU: 0.207 -> 0.229 | Helped |
| Use MGM with weakly labeled synthesized images and joint training instead of MT | Multi-task visual prediction · Tiny-Taskonomy, 50% data | DE-mLoss: 1.595 -> 1.499 | Helped |
| Add synthesized images produced by an off-the-shelf GAN, with oracle pixel-level annotations | Single-task semantic segmentation · Tiny-Taskonomy | mLoss: 0.111 -> 0.148 | Hurt |
| Add joint image generation rather than training the generator separately (ablation) | MGM · NYUv2, 50% data | DE-mABSE: 0.763 -> 0.734 | Helped |
| Add synthesized-image generation (ablation) | MGM · Tiny-Taskonomy, 50% data | DE-mLoss: 1.527 -> 1.499 | Helped |
| Add the self-supervision task (ablation) | MGM · NYUv2, 50% data | SS-mIOU: 0.239 -> 0.251 | Helped |
| Add the refinement network (ablation) | MGM · NYUv2, 50% data | SN-mAD: 0.290 -> 0.273 | Helped |
| Replace SimCLR with reconstruction self-supervision (ablation) | MGM · NYUv2, 50% data | SS-mIOU: 0.251 -> 0.241 | Hurt |
| Replace SimCLR with reconstruction self-supervision (ablation) | MGM · NYUv2, 50% data | DE-mABSE: 0.734 -> 0.768 | Helped |
| Increase model parameters using deeper ResNet-34 baselines (ablation) | ST and MT baselines · NYUv2, 50% data | SS-mIOU: 0.230 -> 0.232; 0.237 -> 0.236 | No clear effect |
Closest papers by meaning
- InstructCV: Instruction-Tuned Text-to-Image Diffusion Models as Vision Generalists (ICLR 2024 · similarity 0.87)
- Diffusion Models for Multi-Task Generative Modeling (ICLR 2024 · similarity 0.87)
- Compositional Scene Understanding through Inverse Generative Modeling (ICML 2025 · similarity 0.86)
- UtilGen: Utility-Centric Generative Data Augmentation with Dual-Level Task Adaptation (NeurIPS 2025 · similarity 0.85)
- Unified Generative and Discriminative Training for Multi-modal Large Language Models (NeurIPS 2024 · similarity 0.85)
- Learning Visual Generative Priors without Text (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.