Research map / Vision-language and multimodal / Human and embodied modeling
Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Design2GarmentCode uses a LoRA-finetuned LLM to write GarmentCode parametric pattern-making programs from text, image, and sketch design inputs, with a pre-trained LMM interpreting the design and a lightweight transformer projector mapping answers to parameters.
Problem
Sewing pattern generation from multi-modal design concepts is hard: uni-modal learned models need large paired datasets and produce oversimplified patterns with self-intersections and wrong stitches, lacking centimeter-level geometric precision.
Approach
Instead of directly generating vector-quantized patterns, an LoRA-finetuned LLM synthesizes GarmentCode parametric programs and design configurations, using a pre-trained multimodal LLM as a design interpreter that answers multiple-choice questions about the input, followed by a learned projector and iterative validation against the design.
Key results
- Text-guided generation achieves 100% simulation success rate versus DressCode's 84%, with 79.83% agreement vs 7.17% for DressCode.
- Image-guided generation achieves 94% simulation success rate and 88.67% agreement versus Sewformer's 65.33% SSR and 3.33% agreement.
- Chamfer distance on image-guided generation is 2.091 versus Sewformer's 9.7, and F-Score is 0.829 versus 0.708.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Design2GarmentCode instead of DressCode | Text-guided sewing pattern generation · Text-guided generation | SSR: 84% -> 100% | Helped |
| Use Design2GarmentCode instead of DressCode | Text-guided sewing pattern generation · Text-guided generation | Agreement: 7.17% -> 79.83% | Helped |
| Use Design2GarmentCode instead of DressCode | Text-guided sewing pattern generation · Text-guided generation | Aesthetic: 9.50% -> 68.17% | Helped |
| Use Design2GarmentCode instead of DressCode | Text-guided sewing pattern generation · Text-guided generation | CLIPScore: 0.2386 -> 0.2438 | Helped |
| Use Design2GarmentCode instead of Sewformer | Image-guided sewing pattern generation · Image-guided generation | SSR: 65.33% -> 94% | Helped |
| Use Design2GarmentCode instead of Sewformer | Image-guided sewing pattern generation · Image-guided generation | Agreement: 3.33% -> 88.67% | Helped |
| Use Design2GarmentCode instead of Sewformer | Image-guided sewing pattern generation · Image-guided generation | Aesthetic: 5.33% -> 77% | Helped |
| Use Design2GarmentCode instead of Sewformer | Image-guided sewing pattern generation · Image-guided generation | F-Score: 0.708 -> 0.829 | Helped |
| Use Design2GarmentCode instead of Sewformer | Image-guided sewing pattern generation · Image-guided generation | CD: 9.7 -> 2.091 | Helped |
Surprising findings
- Pre-trained LLMs already carry foundational pattern-drafting knowledge, producing conventional drafting instructions when prompted.
Limitations
- Cannot substantially alter GarmentCode's underlying structure and logistics, limiting generation quality for out-of-domain garments.
- Relies on GarmentCode's expressiveness and modeling capabilities.
- Evaluation relies on a user study with 30 pattern-makers and a limited in-house set of 150 prompts and 150 images.
Experimental setup
Datasets: GarmentCode programs GarmentCodeData
Benchmarks: 150 text prompts and 150 test images (in-house evaluation set)
Metrics: Simulation Success Rate Agreement Aesthetic CLIPScore F-Score Chamfer Distance number of panels number of edges number of stitches
Baselines: DressCode Sewformer
Builds on: GarmentCode DressCode LoRA Sewformer
Closest papers by meaning
- AIpparel: A Multimodal Foundation Model for Digital Garments (CVPR 2025 · similarity 0.89)
- ChatGarment: Garment Estimation, Generation and Editing via Large Language Models (CVPR 2025 · similarity 0.87)
- DressWild: Feed-Forward Pose-Agnostic Garment Sewing Pattern Generation from In-the-Wild Images (NeurIPS 2026 · similarity 0.85)
- HieraFashDiff: Hierarchical Fashion Design with Multi-stage Diffusion Models (AAAI 2025 · similarity 0.85)
- PICTURE: PhotorealistIC virtual Try-on from UnconstRained dEsigns (CVPR 2024 · similarity 0.84)
- Text-Driven Fashion Image Editing with Compositional Concept Learning and Counterfactual Abduction (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.