Research map / Time series / Foundation models for time series
Text2Data: Low-Resource Data Generation with Textual Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Text2Data, which pretrains a classifier-free diffusion model on all unlabeled data and then finetunes on text-labeled data under a constraint-optimization (lexicographic) objective to control generation and mitigate catastrophic forgetting in low-resource text-to-data generation.
Problem
Text-to-data generation is hard in low-resource modalities (molecules, human motion, time series) where few samples have text labels, causing overfitting and catastrophic forgetting when finetuning diffusion models.
Approach
Use unlabeled data to learn the overall data distribution with an unsupervised (NULL-conditioned) diffusion model, then finetune on labeled pairs with a constraint objective that keeps parameters within the low-loss set from pretraining, solved via a dynamic-gradient lexicographic optimization.
Key results
- On HumanML3D at 40% labels, Text2Data reaches R precision 0.49 vs 0.46 (MDM-finetune) and 0.45 (MDM).
- Text2Data reports average improvements of 19.07% and 58.03% in -log p over EDM-finetune and EDM on molecules.
- On time series, Text2Data achieves lowest MAE for frequency, skewness and mean across most label proportions versus DiffTS and DiffTS-finetune.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Text2Data's unlabeled-data pretraining and constrained finetuning instead of unconstrained MDM finetuning | MDM text-to-motion generation · HumanML3D, 40% paired data | R Precision: 0.46$\pm$0.01 -> 0.49$\pm$0.01 | Helped |
| Use Text2Data's unlabeled-data pretraining and constrained finetuning instead of unconstrained MDM finetuning | MDM text-to-motion generation · HumanML3D, 40% paired data | Multimodal Distance: 5.63$\pm$0.09 -> 5.61$\pm$0.05 | Helped |
| Use Text2Data's unlabeled-data pretraining and constrained finetuning instead of unconstrained DiffTS finetuning | DiffTS time-series generation · Time series, 40% paired data | Skewness MAE: 1.15$\pm$0.24 -> 1.03$\pm$0.11 | Helped |
| Generate pseudo labels for unlabeled molecules with a classifier in a semi-supervised approach (ablation) | EDM-semi · Molecules | controllability MAE: 54.65\% worse | Hurt |
Surprising findings
- MDM-finetune slightly outperforms Text2Data in R precision at small labeled proportions (2%), attributed to milder forgetting with fewer samples.
- EDM-semi (pseudo-labeling semi-supervised baseline) performs worse than the EDM baseline on molecules because pseudo-label errors mislead text alignment.
Limitations
- Relies on a classifier-free diffusion backbone; the approach is presented for diffusion models only.
- Theoretical bound depends on finite parameter set assumption and sub-Gaussian assumptions on noise predictions.
- Time-series and molecule text descriptions are template-generated and refined by GPT-3.5, which may not reflect natural user language.
Experimental setup
Datasets: QM9 HumanML3D Yahoo Finance stock time series
Benchmarks: QM9 HumanML3D Yahoo Finance time series
Metrics: MAE R precision Multimodal distance FID diversity validity molecular stability atom stability negative log-likelihood t-SNE overlap
Baselines: EDM EDM-finetune MDM MDM-finetune DiffTS DiffTS-finetune EDM-semi EDM-DA
Builds on: classifier-free diffusion guidance EDM MDM lexicographic optimization
Closest papers by meaning
- T2I-Adapter: Learning Adapters to Dig Out More Controllable Ability for Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.86)
- Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Following (CVPR 2024 · similarity 0.85)
- CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation (ICLR 2025 · similarity 0.85)
- Progressive Text-to-Image Diffusion with Soft Latent Direction (AAAI 2024 · similarity 0.85)
- Minority-Focused Text-to-Image Generation via Prompt Optimization (CVPR 2025 · similarity 0.85)
- BRIDGE: Bootstrapping Text to Control Time-Series Generation via Multi-Agent Iterative Optimization and Diffusion Modeling (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.