Research map / Large language models / Language, safety and interpretability
Text2Arch: A Dataset for Generating Scientific Architecture Diagrams from Natural Language Descriptions
Shivank Garg, Sankalp Mittal, Manish Gupta
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune DeepSeek-7B on Text2Arch instead of few-shot prompting | DeepSeek-7B · Text2Arch test set | Node F1: 43.5 -> 65.7 | Improved |
| fine-tune Llama-3-8B on Text2Arch instead of few-shot prompting | Llama-3-8B · Text2Arch test set | Node F1: 59.7 -> 31.9 | Hurt |
| fine-tune Qwen2-7B on Text2Arch instead of few-shot prompting | Qwen2-7B · Text2Arch test set | Edge F1: 24.3 -> 13.7 | Hurt |
| fine-tune DeepSeek-7B instead of few-shot prompting | DeepSeek-7B · Text2Arch test set | Edge F1: 16.6 -> 38.0 | Improved |
| fine-tune DeepSeek-7B instead of few-shot prompting | DeepSeek-7B · Text2Arch manual annotation set (99 images) | Edge F1: 20.4 -> 49.1 | Improved |
| use fine-tuned DeepSeek-7B instead of zero-shot GPT-4o | Text2Arch · Text2Arch manual annotation set (99 images) | ROUGE-L: 28.2 -> 55.2 | Improved |
| use fine-tuned DeepSeek-7B instead of DiagramAgent | Text2Arch · Text2Arch test set | ROUGE-L: 42.2 -> 46.8 | Improved |
| use fine-tuned DeepSeek-7B instead of zero-shot GPT-4o by GPT judge | Text2Arch · GPT-4o judged compatibility (0-5) | compatibility score: 2.72 vs 2.68 | No clear effect |
| add GPT-4o refinement on top of detector-based DOT2 to form DOT3 (ablation) | DOT label pipeline · DOT label quality (manual set) | Edge F1: 8.3 -> 51.7 | Improved |
| add detector/OCR structure grounding to GPT extraction (DOT3 vs DOT1) (ablation) | DOT label pipeline · DOT label quality (manual set) | Node F1: 67.5 -> 74.5 | Improved |
| use description Desc3 (retrieved paragraphs plus GPT revision) instead of Desc1 or Desc2 (ablation) | description pipeline · GPT pairwise preference | preference rate: over 90% | Improved |
| train arch classifier with CLIP at learning rate 1e-5 instead of 5e-4 (ablation) | CLIP classifier · Arch vs no-arch test | test accuracy: 0.5533 -> 0.8345 | Improved |
Closest papers by meaning
- From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generation and Editing (CVPR 2025 · similarity 0.84)
- Text-to-Code Generation for Modular Building Layouts in Building Information Modeling (NeurIPS 2025 · similarity 0.84)
- ScImage: How good are multimodal large language models at scientific text-to-image generation? (ICLR 2025 · similarity 0.83)
- SysCaps: Language Interfaces for Simulation Surrogates of Complex Systems (ICLR 2025 · similarity 0.82)
- SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models (NeurIPS 2024 · similarity 0.82)
- AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomy (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
