Research map / Image generation / Text-to-image diffusion
A Unified Image-Dense Annotation Generation Model for Underwater Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TIDE, a text-only model that jointly generates underwater images, depth maps and semantic masks via implicit layout sharing and time adaptive normalization, and uses the synthetic SynTIDE dataset to improve underwater depth estimation and segmentation.
Problem
Underwater dense prediction (depth estimation and semantic segmentation) suffers from scarce large-scale datasets with dense annotations, and prior synthesis methods like Atlantis produce only one annotation type and need depth-map conditions.
Approach
Builds parallel text-to-image, text-to-depth and text-to-mask denoising branches on PixArt-α. Implicit Layout Sharing reuses the text-to-image cross-attention map in the dense-annotation branches, and Time Adaptive Normalization modulates features across modalities with a time-dependent gate, so images and annotations stay consistent using text alone.
Key results
- On Sea-thru D3/D5 with NewCRFs, SynTIDE lowers SIlog from 37.10 (Atlantis) to 22.37 (-14.73) and raises δ1 by 0.36.
- With ViT-Adapter on USIS10K, pre-training on SynTIDE improves mIoU by 2.1% (from 74.6 to 76.7).
- Combining ILS and TAN raises segmentation mIoU in ablation from 36.8 (ILS only) and 36.2 (TAN only) to 42.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune NewCRFs on SynTIDE instead of Atlantis | NewCRFs · Sea-thru D3 and D5 | SIlog: 37.10 -> 22.37 | Helped |
| Fine-tune NewCRFs on SynTIDE instead of Atlantis | NewCRFs · Sea-thru D3 and D5 | δ1: 0.48 -> 0.84 | Helped |
| Fine-tune MIM on SynTIDE instead of Atlantis | MIM · Sea-thru D3 and D5 | SIlog: 37.01 -> 22.49 | Helped |
| Fine-tune MIM on SynTIDE instead of Atlantis | MIM · Sea-thru D3 and D5 | δ1: 0.56 -> 0.85 | Helped |
| Fine-tune PixelFormer on SynTIDE instead of Atlantis | PixelFormer · SQUID | SIlog: 21.34 -> 19.08 | Helped |
| Pre-train with SynTIDE and train with real data | ViT-Adapter · USIS10K | mIoU: 74.6 -> 76.7 | Helped |
| Add TAN to ILS (ablation) | TIDE · USIS10K | mIoU: 36.8 -> 42.1 | Helped |
| Add ILS to TAN (ablation) | TIDE · USIS10K | mIoU: 36.2 -> 42.1 | Helped |
| Apply ILS and TAN in the first half rather than the second half (ablation) | TIDE · Sea-thru D3 and D5 | SIlog: 44.86 -> 22.06 | Helped |
| Increase samples per caption from 1 to 10 (ablation) | TIDE synthetic data · Sea-thru D3 and D5 | SIlog: 23.49 -> 22.37 | Helped |
| Increase samples per caption from 1 to 10 (ablation) | TIDE synthetic data · USIS10K | mIoU: 55.3 -> 64.2 | Helped |
Surprising findings
- Applying ILS and TAN in the second half of the transformer performs worse than the first half, which the authors attribute to layout mismatch.
- Increasing synthetic samples per caption yields no substantial gain for depth estimation, while segmentation gains flatten after N=6.
Limitations
- Cannot generate instance-level semantic masks from text.
- Challenging conditions such as low light and turbidity still have room for improvement.
- Depth gains are not consistent across all metrics; some metrics worsen versus Atlantis on certain models.
Experimental setup
Datasets: SUIM UIIS USIS10K Sea-thru D3/D5 SQUID
Benchmarks: Sea-thru D3/D5 SQUID UIIS USIS10K
Metrics: SIlog A.Rel log10 RMSE S.Rel RMSElog δ1 δ2 δ3 mIoU
Baselines: Atlantis AdaBins NewCRFs PixelFormer MIM Segformer Mask2former ViT-Adapter
Builds on: PixArt-α Latent Diffusion Model LoRA Atlantis
Closest papers by meaning
- NAUTILUS: A Large Multimodal Model for Underwater Scene Understanding (NeurIPS 2025 · similarity 0.86)
- SeafloorAI: A Large-scale Vision-Language Dataset for Seafloor Geological Survey (NeurIPS 2024 · similarity 0.86)
- Atlantis: Enabling Underwater Depth Estimation with Stable Diffusion (CVPR 2024 · similarity 0.85)
- Diving into Underwater: Segment Anything Model Guided Underwater Salient Instance Segmentation and A Large-scale Dataset (ICML 2024 · similarity 0.85)
- WebUOT-1M: Advancing Deep Underwater Object Tracking with A Million-Scale Benchmark (NeurIPS 2024 · similarity 0.83)
- Plenodium: Underwater 3D Scene Reconstruction with Plenoptic Medium Representation (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.