Research map / Image generation / Fast diffusion sampling

Doctor Approved: Generating Medically Accurate Skin Disease Images through AI-Expert Feedback

NeurIPS 2025 · Image generation › Fast diffusion sampling · approach: Diffusion models as perception tools · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MAGIC, a semi-automated framework that uses GPT-4o as an evaluator of dermatologist-written checklists to align a fine-tuned Stable Diffusion model (via RFT or DPO) so it generates medically accurate skin disease images for classifier data augmentation.

Problem

Diffusion models often generate medically inaccurate skin disease images, and existing feedback-based alignment (RLHF, DPO) needs robust reward models or costly expert annotation, limiting their use for augmenting scarce medical data.

Approach

Expert-written attribute-level checklists (location, lesion type, shape/size, color, texture) are scored by an MLLM (GPT-4o) on synthetic image pairs; the binary outcomes serve as preference labels for DPO (or reward-weighted fine-tuning) of an image-to-image Stable Diffusion pipeline, so the MLLM evaluates only simple closed-form visual criteria.

Key results

• MAGIC-DPO raised ResNet18 accuracy on the 20-condition Fitzpatrick17k task from 29.31% to 38.33% (+9.02%) and DINOv2 from 49.89% to 55.01% (+5.12%).
• In the 10% few-shot setting (310 real images), MAGIC-DPO reached 37.39% accuracy (+10.94%), and MAGIC-A with unlabeled data reached 40.34% (+13.89%).
• Expert evaluation: 55.5% of MAGIC-DPO images met 3 or more of 5 criteria, versus 18.5% for fine-tuned I2I and 2.0% for T2I.

Measured effects reported in the paper

Surprising findings

• Standard fine-tuned text-to-image generation degraded classifier accuracy (ResNet18 -3.74%, DINOv2 -2.16%), while the I2I variant helped.
• MedGemma-4B used as the evaluator achieved results close to GPT-4o (DINOv2 54.19% vs 55.01%), suggesting a smaller open-source judge is viable.

Limitations

• Performance is bounded by the quality of expert checklists and the capability of the MLLM evaluator (GPT-4o), which can fail on subtle textures, color nuances, and 3D cues.
• Evaluated only on Fitzpatrick17k, SCIN, and PAD-UFES-20; the SCIN augmented test set gave suboptimal results attributed to label noise.
• Synthetic-to-real domain shift requires careful tuning of the synthetic data ratio; purely synthetic training overfits.

Experimental setup

Datasets: Fitzpatrick17k (20-condition subset) SCIN PAD-UFES-20

Benchmarks: Fitzpatrick17k 20-condition classification SCIN PAD-UFES-20 few-shot Fitzpatrick17k

Metrics: accuracy F1 precision recall FID criteria satisfaction rate dermatologist alignment score

Baselines: Real-only training T2I fine-tuned Stable Diffusion I2I fine-tuned Stable Diffusion MAGIC-RFT Coarse checklist MAGIC

Builds on: Direct Preference Optimization Stable Diffusion Textual Inversion LoRA Diffusion-DPO

Closest papers by meaning

• DermaCon-IN: A Multiconcept-Annotated Dermatological Image Dataset of Indian Skin Disorders for Clinical AI Research (NeurIPS 2025 · similarity 0.87)
• MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment (AAAI 2024 · similarity 0.85)
• QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Training (NeurIPS 2025 · similarity 0.84)
• Patho-R1: A Multimodal Reinforcement Learning-Based Pathology Expert Reasoner (AAAI 2026 · similarity 0.83)
• MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level Precision (AAAI 2026 · similarity 0.83)
• How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images (ICLR 2026 · similarity 0.83)