Generative AI for Synthetic Data Across Multiple Medical Modalities: A Systematic Review
This systematic review covers generative models (GANs, VAEs, diffusion models, and LLMs) used to synthesize medical data, spanning imaging (dermoscopic, mammographic, ultrasound, CT, MRI, X-ray), text, time-series, and tabular EHR data. Unlike prior narrowly focused reviews, it encompasses a broad array of modalities, offering insight into current and future applications across synthesis, generation techniques, and evaluation methods, alongside a GitHub repository as a living resource. The multi-modality scope surfaces cross-modality insights unavailable in single-modality surveys; core generative techniques are transferable, but synthesis methods often lack sufficient integration of patient-specific context and clinical knowledge. Findings are structured around three themes, synthesis applications, generation techniques, and evaluation methods, revealing an absence of standardized benchmarks and a need for large-scale, privacy-aware validation and cross-team benchmarking.
Abstract
This paper presents a comprehensive systematic review of generative models (GANs, VAEs, DMs, and LLMs) used to synthesize various medical data types, including imaging (dermoscopic, mammographic, ultrasound, CT, MRI, and X-ray), text, time-series, and tabular data (EHR). Unlike previous narrowly focused reviews, our study encompasses a broad array of medical data modalities and explores various generative models. Our aim is to offer insights into their current and future applications in medical research, particularly in the context of synthesis applications, generation techniques, and evaluation methods, as well as providing a GitHub repository as a dynamic resource for ongoing collaboration and innovation.
Key contributions include a broad, multi-modality scope that identifies cross-modality insights and opportunities unavailable in single-modality surveys. While core generative techniques are transferable, we find that synthesis methods often lack sufficient integration of patient-specific context, clinical knowledge, and modality-specific requirements tailored to the unique characteristics of medical data. Conditional models leveraging textual conditioning and multimodal synthesis remain underexplored but offer promising directions for innovation.
Our findings are structured around three themes: (1) Synthesis applications, highlighting clinically valid synthesis applications and significant gaps in using synthetic data beyond augmentation, such as for validation and evaluation; (2) Generation techniques, identifying gaps in personalization and cross-modality innovation; and (3) Evaluation methods, revealing the absence of standardized benchmarks, the need for large-scale validation, and the importance of privacy-aware, clinically relevant evaluation frameworks. These findings emphasize the need for benchmarking and comparative studies to promote openness and collaboration.