Research map / Vision-language and multimodal / Human and embodied modeling
AIpparel: A Multimodal Foundation Model for Digital Garments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes LLaVA 1.5-7B on a new multimodal sewing-pattern dataset (GCD-MM, ~120k garments) with a compact sewing-pattern tokenizer and regression heads to generate and edit sewing patterns from text, images, and edit instructions.
Problem
Generating and editing sewing patterns from multiple input modalities (text, images, editing instructions) is hard because no large multimodal sewing-pattern dataset exists and existing single-modal models do not scale to multimodal inputs or represent complex patterns compactly.
Approach
Represent sewing patterns as a compact sequence of drawing-command-style special tokens (about 250 tokens per pattern vs. >30k for DressCode), predict continuous vertex and transform parameters with small regression heads on the LMM's hidden states, and fine-tune an LLaVA-based LMM on an annotated multimodal dataset.
Key results
- Image-to-garment on GCD-MM: AIpparel reaches 85.2 #Panel Acc vs 79.4 for SewFormer-FT, with Panel L2 5.4 vs 12.3 cm.
- Multimodal generation on 100 garments: AIpparel Accuracy 59.0 vs 10.3 for SewFormer-FT† and 0.6 for DressCode†.
- Ablation on DressCode data: the tokenizer cuts inference time to 2.1 s vs 52.2 s for DressCode (about 25x faster) while raising Accuracy from 38.4 to 85.0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AIpparel instead of SewFormer | Image-to-garment prediction · SewFactory | Panel L2: 3.3 -> 2.8 | Helped |
| Use AIpparel instead of SewFormer-FT | Image-to-garment prediction · GCD-MM | Panel L2: 12.3 -> 5.4 | Helped |
| Use AIpparel instead of SewFormer-FT† | Multimodal garment prediction · Multimodal garment prediction | Accuracy: 10.3 -> 59.0 | Helped |
| Use AIpparel instead of DressCode* | Sewing pattern editing · Sewing pattern editing | Accuracy: 37.0 -> 83.4 | Helped |
| Add regression loss for continuous parameters instead of cross-entropy-only prediction (ablation) | AIpparel tokenizer and training objective · DressCode text-to-garment | Accuracy: 79.0 -> 85.0 | Helped |
| Add regression loss for continuous parameters instead of cross-entropy-only prediction (ablation) | AIpparel tokenizer and training objective · DressCode text-to-garment | Panel L2: 7.2 -> 6.1 | Helped |
| Add regression loss for continuous parameters instead of cross-entropy-only prediction (ablation) | AIpparel tokenizer and training objective · DressCode text-to-garment | Time: 3.4s -> 2.1s | Helped |
| Replace DressCode’s tokenizer with the proposed tokenizer (ablation) | AIpparel · DressCode text-to-garment | Accuracy: +46.6 | Helped |
Limitations
- Representation limited to garments representable by manifold surfaces; pockets and other non-manifold features not supported.
- Fabrication constraints (physical and material) not considered in pattern prediction.
- Inherits misinformation and bias risks from the pre-trained LLaVA model.
Experimental setup
Datasets: GarmentCodeData GCD-MM SewFactory DressCode dataset
Benchmarks: GCD-MM SewFactory DressCode dataset
Metrics: Accuracy Panel L2 #Panel Acc #Edge Acc #Stitch Acc Rot L2 Transl L2 inference time
Baselines: SewFormer SewFormer-FT DressCode DressCode* Sewformer*
Builds on: LLaVA 1.5 GarmentCode GarmentCodeData SewFormer DressCode
Closest papers by meaning
- ChatGarment: Garment Estimation, Generation and Editing via Large Language Models (CVPR 2025 · similarity 0.90)
- Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis (CVPR 2025 · similarity 0.89)
- AnyDressing: Customizable Multi-Garment Virtual Dressing via Latent Diffusion Models (CVPR 2025 · similarity 0.87)
- FashionTailor: Controllable Clothing Editing for Human Images with Appearance Preserving (AAAI 2025 · similarity 0.87)
- DressWild: Feed-Forward Pose-Agnostic Garment Sewing Pattern Generation from In-the-Wild Images (NeurIPS 2026 · similarity 0.86)
- DreamFit: Garment-Centric Human Generation via a Lightweight Anything-Dressing Encoder (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.