Research map / Large language models / Retrieval and knowledge
QGen Studio: An Adaptive Question-Answer Generation, Training and Evaluation Platform
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents QGen Studio, a platform for LLM-driven QA dataset generation, LoRA fine-tuning via MLX, dataset visualization with quality metrics, and side-by-side model comparison.
Problem
Niche domains lack tailored QA datasets, and existing tools for synthetic QA dataset creation, quality inspection, and model training/comparison limit user control over generation and make datasets hard to understand.
Approach
An interactive end-to-end studio that combines document upload, few-shot example prompting, LLM-based QA generation with n-gram/TF-IDF/cosine quality metrics, a dataset viewer that highlights source context, LoRA fine-tuning on Apple Silicon via MLX, and a model explorer for side-by-side comparison.
Key results
- No quantitative results are reported; the paper describes the platform's features and pipeline only.
Limitations
- Model training is currently limited to Apple Silicon users via MLX.
- No quantitative evaluation of the platform or of the generated dataset quality is reported.
- Code is not yet released; the studio will be open-sourced 'soon'.
Experimental setup
Datasets: user-uploaded documents (JSON, PDFs, URLs converted via Docling)
Metrics: BLEU ROUGE METEOR TF-IDF cosine similarity
Builds on: LoRA Docling MLX
Closest papers by meaning
- Putting People in LLMs’ Shoes: Generating Better Answers via Question Rewriter (AAAI 2025 · similarity 0.87)
- QA-Calibration of Language Model Confidence Scores (ICLR 2025 · similarity 0.87)
- Rethinking Generative Large Language Model Evaluation for Semantic Comprehension (ICML 2024 · similarity 0.86)
- How Teachers Can Use Large Language Models and Bloom’s Taxonomy to Create Educational Quizzes (AAAI 2024 · similarity 0.86)
- A Semantic-Sampling Framework for Evaluating Calibration in Open-Ended Question Answering (NeurIPS 2026 · similarity 0.86)
- Reliable and Efficient Amortized Model-based Evaluation (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.