AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home CAAI Artificial Intelligence Research Article
PDF (5.4 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Article | Open Access

ADGAN: Adaptive Domain Medical Image Synthesis Based on Generative Adversarial Networks

Liming Xu1,2( ), Yanrong Lei1, Bochuan Zheng1, Jiancheng Lv2, Weisheng Li3
School of Computer Science, China West Normal University, Nanchong 637002, China
College of Computer Science and Technology, Sichuan University, Chengdu 130012, China
College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
Show Author Information

Abstract

Multimodal medical imaging of human pathological tissues provides comprehensive information to assist in clinical diagnosis. However, due to the high cost of imaging, physiological incompatibility, and the harmfulness of radioactive tracers, multimodal medical image data remains scarce. Currently, cross-modal medical synthesis methods can generate desired modal images from existing modal images. However, most existing methods are limited to specific domains. This paper proposes an Adaptive Domain Medical Image Synthesis Method based on Generative Adversarial Networks (ADGAN) to address this issue. ADGAN achieves multidirectional medical image synthesis and ensures pathological consistency by constructing a single generator to learn the latent shared representation of multiple domains. The generator employs dense connections in shallow layers to preserve edge details and incorporates auxiliary information in deep layers to retain pathological features. Additionally, spectral normalization is introduced into the discriminator to control discriminative performance and indirectly enhance the image synthesis ability of the generator. Theoretically, it can be proved that the proposed method can be trained quickly, and spectral normalization contributes to adaptive and multidirectional synthesis. In practice, comparing with recent state-of-the-art methods, ADGAN achieves average increments of 4.7% SSIM, 6.7% MSIM, 7.3% PSNR, and 9.2% VIF.

Keywords

generative adversarial networksmultimodal medical imagesadaptive domainlatent shared representation

References

[1]

M. J. Willemink, W. A. Koszek, C. Hardell, J. Wu, D. Fleischmann, H. Harvey, L. R. Folio, R. M. Summers, D. L. Rubin, and M. P. Lungren, Preparing medical imaging data for machine learning, Radiology, vol. 295, no. 1, pp. 4–15, 2020.
Crossref Google Scholar
[2]

L. Xu, X. Zeng, H. Zhang, W. Li, J. Lei, and Z. Huang, BPGAN: Bidirectional CT-to-MRI prediction using multi-generative multi-adversarial nets with spectral normalization and localization, Neural Netw., vol. 128, pp. 82–96, 2020.
Crossref Google Scholar
[3]

L. Xu, H. Zhang, L. Song, and Y. Lei, Bi-MGAN: Bidirectional T1-to-T2 MRI images prediction using multi-generative multi-adversarial nets, Biomed. Signal Process. Control, vol. 78, p. 103994, 2022.
Crossref Google Scholar
[4]

A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet classification with deep convolutional neural networks, Commun. ACM, vol. 60, no. 6, pp. 84–90, 2017.
Crossref Google Scholar
[5]

X. Han, MR-based synthetic CT generation using a deep convolutional neural network method, Med. Phys., vol. 44, no. 4, pp. 1408–1419, 2017.
Crossref Google Scholar
[6]

M. F. Spadea, G. Pileggi, P. Zaffino, P. Salome, C. Catana, D. Izquierdo-Garcia, F. Amato, and J. Seco, Deep convolution neural network (DCNN) multiplane approach to synthetic CT generation from MR images—Application in brain Proton Therapy, Int. J. Radiat. Oncol., vol. 105, no. 3, pp. 495–503, 2019.
Crossref Google Scholar
[7]
J. Long, E. Shelhamer, and T. Darrell, Fully convolutional networks for semantic segmentation, in Proc. 2015 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015, pp. 3431–3440.
Crossref
[8]

A. Ben-Cohen, E. Klang, S. P. Raskin, S. Soffer, S. Ben-Haim, E. Konen, M. M. Amitai, and H. Greenspan, Cross-modality synthesis from CT to PET using FCN and GAN networks for improved automated lesion detection, Eng. Appl. Artif. Intell., vol. 78, pp. 186–194, 2019.
Crossref Google Scholar
[9]

A. Chartsias, T. Joyce, M. V. Giuffrida, and S. A. Tsaftaris, Multimodal MR synthesis via modality-invariant latent representation, IEEE Trans. Med. Imaging, vol. 37, no. 3, pp. 803–814, 2018.
Crossref Google Scholar
[10]
G. Li, L. Bai, C. Zhu, E. Wu, and R. Ma, A novel method of synthetic CT generation from MR images based on convolutional neural networks, in Proc. 2018 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), Beijing, China, 2018, pp. 1–5.
Crossref
[11]

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, Generative adversarial networks, Commun. ACM, vol. 63, no. 11, pp. 139–144, 2020.
Crossref Google Scholar
[12]

L. Lan, L. You, Z. Zhang, Z. Fan, W. Zhao, N. Zeng, Y. Chen, and X. Zhou, Generative adversarial networks and its applications in biomedical informatics, Front. Public Health, vol. 8, p. 164, 2020.
Crossref Google Scholar
[13]

C. Tian, Y. Yuan, S. Zhang, C. W. Lin, W. Zuo, and D. Zhang, Image super-resolution with an enhanced group convolutional neural network, Neural Netw., vol. 153, no. 1, pp. 373–385, 2022.
Crossref Google Scholar
[14]

X. Yang, Y. Lin, Z. Wang, X. Li, and K. T. Cheng, Bi-modality medical image synthesis using semi-supervised sequential generative adversarial networks, IEEE J. Biomed. Health Inform., vol. 24, no. 3, pp. 855–865, 2020.
Crossref Google Scholar
[15]
T. Zhang, H. Fu, Y. Zhao, J. Cheng, M. Guo, Z. Gu, B. Yang, Y. Xiao, S. Gao, and J. Liu, SkrGAN: Sketching-rendering unconditional generative adversarial networks for medical image synthesis, in Proc. 22nd Int. Conf. Medical Image Computing and Computer Assisted Intervention, Shenzhen, China, 2019, pp. 777–785.
Crossref
[16]
J. Wang, Y. Zhao, J. H. Noble, and B. M. Dawant, Conditional generative adversarial networks for metal artifact reduction in CT images of the ear, in Proc. 21st Int. Conf. Medical Image Computing and Computer Assisted Intervention, Granada, Spain, 2018, pp. 3–11.
Crossref
[17]

M. Yurt, S. U. Dar, A. Erdem, E. Erdem, K. K. Oguz, and T. Çukur, mustGAN: Multi-stream generative adversarial networks for MR image synthesis, Med. Image Anal., vol. 70, p. 101944, 2021.
Crossref Google Scholar
[18]

N. Xiao, X. Xiao, Y. Qiang, K. Li, S. Li, and J. Lian, Longitudinal prediction of lung tumor based on conditional adversarial spatiotemporal encoder (in Chinese), J. Softw., vol. 34, no. 9, pp. 4392–4406, 2023.
Crossref Google Scholar
[19]

B. Yu, L. Zhou, L. Wang, Y. Shi, J. Fripp, and P. Bourgeat, Ea-GANs: Edge-aware generative adversarial networks for cross-modality MR image synthesis, IEEE Trans. Med. Imaging, vol. 38, no. 7, pp. 1750–1762, 2019.
Crossref Google Scholar
[20]

Y. Luo, D. Nie, B. Zhan, Z. Li, X. Wu, J. Zhou, Y. Wang, and D. Shen, Edge-preserving MRI image synthesis via adversarial network with iterative multi-scale fusion, Neurocomputing, vol. 452, pp. 63–77, 2021.
Crossref Google Scholar
[21]

B. Zhan, D. Li, X. Wu, J. Zhou, and Y. Wang, Multi-modal MRI image synthesis via GAN with multi-scale gate mergence, IEEE J. Biomed. Health Inform., vol. 26, no. 1, pp. 17–26, 2022.
Crossref Google Scholar
[22]

Y. Luo, L. Zhou, B. Zhan, Y. Fei, J. Zhou, Y. Wang, and D. Shen, Adaptive rectification based adversarial network with spectrum constraint for high-quality PET image synthesis, Med. Image Anal., vol. 77, p. 102335, 2022.
Crossref Google Scholar
[23]

S. Ding and J. Lyu, High resolution dermoscopy image synthesis method with pix2pixHD (in Chinese), J. Comput. -Aided Des. Comput. Graphics, vol. 32, no. 11, pp. 1795–1803, 2020.
Google Scholar
[24]

C. Wang, G. Yang, G. Papanastasiou, S. A. Tsaftaris, D. E. Newby, C. Gray, G. Macnaught, and T. J. MacGillivray, DiCyc: GAN-based deformation invariant cross-domain information fusion for medical image synthesis, Inf. Fusion, vol. 67, pp. 147–160, 2021.
Crossref Google Scholar
[25]
J. Y. Zhu, T. Park, P. Isola, and A. A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, in Proc. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 2242–2251.
Crossref
[26]
Z. Zhang, L. Yang, and Y. Zheng, Translating and segmenting multimodal medical volumes with cycle- and shape-consistency generative adversarial network, in Proc. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 9242–9251.
Crossref
[27]
Y. Pan, M. Liu, C. Lian, T. Zhou, Y. Xia, and D. Shen, Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis, in Proc. 21st Int. Conf. Medical Image Computing and Computer Assisted Intervention, Granada, Spain, 2018, pp. 455–463.
Crossref
[28]
G. Zeng and G. Zheng, Hybrid generative adversarial networks for deep MR to CT synthesis using unpaired data, in Proc. 22nd Int. Conf. Medical Image Computing and Computer Assisted Intervention, Shenzhen, China, 2019, pp. 759–767.
Crossref
[29]

S. U. Dar, M. Yurt, L. Karacan, A. Erdem, E. Erdem, and T. Cukur, Image synthesis in multi-contrast MRI with conditional generative adversarial networks, IEEE Trans. Med. Imaging, vol. 38, no. 10, pp. 2375–2388, 2019.
Crossref Google Scholar
[30]

S. Hu, B. Lei, S. Wang, Y. Wang, Z. Feng, and Y. Shen, Bidirectional mapping generative adversarial networks for brain MR to PET synthesis, IEEE Trans. Med. Imaging, vol. 41, no. 1, pp. 145–157, 2022.
Crossref Google Scholar
[31]
Y. Choi, M. Choi, M. Kim, J. W. Ha, S. Kim, and J. Choo, StarGAN: unified generative adversarial networks for multi-domain image-to-image translation, in Proc. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 8789–8797.
Crossref
[32]
Y. Choi, Y. Uh, J. Yoo, and J. W. Ha, StarGAN v2: Diverse image synthesis for multiple domains, in Proc. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 8185–8194.
Crossref
[33]

C. J. Willmott and K. Matsuura, Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance, Clim. Res., vol. 30, pp. 79–82, 2005.
Crossref Google Scholar
[34]
P. Y. Kao, T. Ngo, A. Zhang, J. W. Chen, and B. S. Manjunath, Brain tumor segmentation and tractographic feature extraction from structural MR images for overall survival prediction, in Proc. 4th Int. Workshop Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Granada, Spain, 2018, pp. 128–141.
Crossref
[35]
C. Tian, Y. Yuan, S. Zhang, C. W. Lin, W. Zuo, and D. Zhang, Image super-resolution with an enhanced group convolutional neural network, arXiv preprint arXiv: 2204.13620, 2022.
Crossref
[36]

Q. Zhang, J. Xiao, C. Tian, J. Xu, S. Zhang, and C. W. Lin, A parallel and serial denoising network, Expert Syst. Appl., vol. 231, p. 120628, 2023.
Crossref Google Scholar
CAAI Artificial Intelligence Research
Volume 3,
2024
Article number: 9150035
DOI: 10.26599/AIR.2024.9150035
Cite this article:
Xu L, Lei Y, Zheng B, et al. ADGAN: Adaptive Domain Medical Image Synthesis Based on Generative Adversarial Networks. CAAI Artificial Intelligence Research, 2024, 3: 9150035. https://doi.org/10.26599/AIR.2024.9150035
Part of a topical collection:
ChinaMM 2023

685

Views

117

Downloads

0

Crossref

Altmetrics
Received: 08 August 2023
Revised: 28 March 2024
Accepted: 11 April 2024
Published: 12 June 2024
© The author(s) 2024.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return