AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (10.1 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

SUNet++: A Deep Network with Channel Attention for Small-Scale Object Segmentation on 3D Medical Images

College of Computer Science and Technology, Harbin Engineering University, Harbin 150001, China
Show Author Information

Abstract

As a deep learning network with an encoder-decoder architecture, UNet and its series of improved versions have been widely used in medical image segmentation with great applications. However, when used to segment targets in 3D medical images such as magnetic resonance imaging (MRI), computed tomography (CT), these models do not model the relevance of images in vertical space, resulting in poor accurate analysis of consecutive slices of the same patient. On the other hand, the large amount of detail lost during the encoding process makes these models incapable of segmenting small-scale tumor targets. Aiming at the scene of small-scale target segmentation in 3D medical images, a fully new neural network model SUNet++ is proposed on the basis of UNet and UNet++. SUNet++ improves the existing models mainly in three aspects: 1) the modeling strategy of slice superposition is used to thoroughly excavate the three dimensional information of the data; 2) by adding an attention mechanism during the decoding process, small scale targets in the picture are retained and amplified; 3) in the up-sampling process, the transposed convolution operation is used to further enhance the effect of the model. In order to verify the effect of the model, we collected and produced a dataset of hyperintensity MRI liver-stage images containing over 400 cases of liver nodules. Experimental results on both public and proprietary datasets demonstrate the superiority of SUNet++ in small-scale target segmentation of three-dimensional medical images.

Keywords

3D medical imagessmall-scale targetsegmentationattention mechanism

References

[1]
J. Long, E. Shelhamer, and T. Darrell, Fully convolutional networks for semantic segmentation. in Proc. 2015 IEEE Conf.on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015, pp. 3431–3440, 2015.
Crossref Google Scholar
[2]
O. Ronneberger, P. Fischer, and T. Brox, U-Net: Convolutional networks for biomedical image segmentation, in Proc. 18th Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015, pp. 234–241.
Crossref Google Scholar
[3]
L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, Semantic image segmentation with deep convolutional nets and fully connected CRFs, presented at the 3rd Int. Conf. on Learning Representations, San Diego, CA, USA, 2015.
Google Scholar
[4]
H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, Pyramid scene parsing network. in Proc. 2017 IEEE Conf. on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 6230–6239.
Crossref Google Scholar
[5]
R. Girshick, J. Donahue, T. Darrell, and J. Malik, Rich feature hierarchies for accurate object detection and semantic segmentation, in Proc. 2014 IEEE Conf. on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014, pp. 580–587, 2014.
Crossref Google Scholar
[6]
S. Ren, K. He, R. Girshick, and J. Sun, Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, 2017.
Crossref Google Scholar
[7]
Z. Zhou, M. R. Siddiquee, N. Tajbakhsh, and J. Liang, UNet++: A nested U-net architecture for medical image segmentation, in Proc. 4th Int. Workshop on Deep Learning in Medical Image Analysis, Granada, Spain, 2018, pp. 3–11.
Crossref Google Scholar
[8]
F. Milletari, N. Navab, and S. A. Ahmadi, V-Net: Fully convolutional neural networks for volumetric medical image segmentation, in Proc. 4th Int. Conf. on 3D Vision (3DV), Stanford, CA, USA, 2016, pp. 565–571.
Crossref Google Scholar
[9]
Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, 3D U-Net: Learning dense volumetric segmentation from sparse annotation, in Proc. 19th Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, Athens, Greece, 2016, pp. 424–432.
Crossref Google Scholar
[10]
Z. Cai, X. Zheng, and J. Yu, A differential-private framework for urban traffic flows estimation via taxi companies, IEEE Trans. Industr. Inform., vol. 15, no. 12, pp. 6492–6499, 2019.
Crossref Google Scholar
[11]
Y. Liang, Z. Cai, J. Yu, Q. Han, and Y. Li, Deep learning based inference of private information using embedded sensors in smart devices, IEEE Network, vol. 32, no. 4, pp. 8–14, 2018.
Crossref Google Scholar
[12]
A. Alansary, K. Kamnitsas, A. Davidson, R. Khlebnikov, M. Rajchl, C. Malamateniou, M. Rutherford, J. V. Hajnal, B. Glocker, D. Rueckert, et al., Fast fully automatic segmentation of the human placenta from motion corrupted MRI, in Proc. 19th Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, Athens, Greece, 2016, pp. 589–597.
Crossref Google Scholar
[13]
M. Shakeri, S. Tsogkas, E. Ferrante, S. Lippe, S. Kadoury, N. Paragios, and I. Kokkinos, Sub-cortical brain structure segmentation using F-CNN’S, in Proc. 13th Int. Sympos. on Biomedical Imaging (ISBI), Prague, Czech Republic, 2016, pp. 269–272.
Crossref Google Scholar
[14]
P. Moeskops, J. M. Wolterink, B. H. M. van der Velden, K. G. A. Gilhuijs, T. Leiner, M. A. Viergever, and I. Išgum, Deep learning for multi-task medical image segmentation in multiple modalities, in Proc. 19th Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, Athens, Greece, 2016, pp. 478–486.
Crossref Google Scholar
[15]
X. Zhou, T. Ito, R. Takayama, S. Wang, T. Hara, and H. Fujita, Three-dimensional CT image segmentation by combining 2D fully convolutional network with 3D majority voting, in Proc. 1st Int. Workshop on Deep Learning in Medical Image Analysis, Athens, Greece, 2016, pp. 111–120.
Crossref Google Scholar
[16]
R. Korez, B. Likar, F. Pernuš, and T. Vrtovec, Model-based segmentation of vertebral bodies from MR images with 3D CNNs, in Proc. 19th Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, Athens, Greece, 2016, pp. 433–441.
Crossref Google Scholar
[17]
M. Drozdzal, E. Vorontsov, G. Chartrand, S. Kadoury, and C. Pal, The importance of skip connections in biomedical image segmentation, in Proc. 1st Int. Workshop on Deep Learning in Medical Image Analysis, Athens, Greece, 2016, pp. 179–187.
Crossref Google Scholar
[18]
K. Kamnitsas, C. Ledig, V. F. J. Newcombe, J. P. Simpson, A. D. Kane, D. K. Menon, D. Rueckert, and B. Glocker, Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation, Med. Image Anal., vol. 36, pp. 61–78, 2017.
Crossref Google Scholar
[19]
M. Ghafoorian, N. Karssemeijer, T. Heskes, I. W. M. van Uder, F. E. de Leeuw, E. Marchiori, B. van Ginneken, and B. Platel, Non-uniform patch sampling with deep convolutional neural networks for white matter hyperintensity segmentation. in Proc. 13th Int. Sympos. on Biomedical Imaging, Prague, Czech Republic, 2016, pp. 1414–1417.
Crossref Google Scholar
[20]
T. Brosch, L. Y. W. Tang, Y. Yoo, D. K. B. Li, A. Traboulsee, and R. Tam, Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation, IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1229–1239, 2016.
Crossref Google Scholar
[21]
H. Huang, L. Lin, R. Tong, H. Hu, Q. Zhang, Y. Iwamoto, X. Han, Y. W. Chen, and J. Wu, UNet 3+: A full-scale connected UNet for medical image segmentation, in Proc. 2020 IEEE Int. Conf. on Acoustics, Speech and Signal Processing, Barcelona, Spain, 2020, pp. 1055–1059.
Crossref Google Scholar
[22]
A. E. Kavur, N. S. Gezer, M. Barış S. Aslan, P. H. Conze, V. Groza, D. D. Pham, S. Chatterjee, P. Ernst, S. Özkan, et al., CHAOS challenge-combined (CT-MR) healthy abdominal organ segmentation, Med. Image Anal., vol. 69, p. 101950, 2021.
Crossref Google Scholar
[23]
A. Andreopoulos and J. K. Tsotsos, Efficient and generalizable statistical models of shape and appearance for analysis of cardiac MRI, Med. Image Anal., vol. 12, no. 3, pp. 335–357, 2008.
Crossref Google Scholar
[24]
O. Bernard, A. Lalande, C. Zotti, F. Cervenansky, X. Yang, P. A. Heng, I. Cetin, K. Lekadir, O. Camara, M. A. G. Ballester, et al., Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: Is the problem solved? IEEE Trans. Med. Imaging, vol. 37, no. 11, pp. 2514–2525, 2018.
Crossref Google Scholar
[25]
G. Litjens, R. Toth, W. van de Ven, C. Hoeks, S. Kerkstra, B. van Ginneken, G. Vincent, G. Guillard, N. Birbeck, J. Zhang, et al., Evaluation of prostate segmentation algorithms for MRI: The PROMISE12 challenge, Med. Image Anal., vol. 18, no. 2, pp. 359–373, 2014.
Crossref Google Scholar
[26]
P. Christ, LiTS-liver tumor segmentation challenge (LiTS17), https://academictorrents.com/details/27772adef6f563a1ecc0ae19a528b956e6c803ce, 2017.
[27]
Z. Cai, Z. He, X. Guan, and Y. Li, Collective data-sanitization for preventing sensitive information inference attacks in social networks, IEEE Trans. Dependable Secure Comput., vol. 15, no. 4, pp. 577–590, 2018.
Google Scholar
[28]
Z. Cai and X. Zheng, A private and efficient mechanism for data uploading in smart cyber-physical systems, IEEE Trans. Netw. Sci. Eng., vol. 7, no. 2, pp. 766–775, 2020.
Crossref Google Scholar
[29]
X. Zheng and Z. Cai, Privacy-preserved data sharing towards multiple parties in industrial IoTs. IEEE J. Sel. Areas Commun., vol. 38, no. 5, pp. 968–979, 2020.
Crossref Google Scholar
[30]
Z. Cai, Z. Xiong, H. Xu, P. Wang, W. Li, and Y. Pan, Generative adversarial networks: A survey toward private and secure applications, ACM Comput. Surv., vol. 54, no. 6, p. 132, 2022.
Crossref Google Scholar
[31]
K. Simanyan and A. Zisserman, Very deep convolutional network for large-scale image recognition, arXiv preprint arXiv: 1409.1556, 2015.
Google Scholar
[32]
J. Wang, Z. Cai, and J. Yu, Achieving personalized k-anonymity-based content privacy for autonomous vehicles in CPS, IEEE Trans. Industr. Inform., vol. 16, no. 6, pp. 4242–4251, 2020.
Crossref Google Scholar
[33]
X. Zheng, Z. Cai, J. Li, and H. Gao, Location-privacy-aware review publication mechanism for local business service systems, in Proc. 2017 IEEE Conf. on Computer Communications, Atlanta, GA, USA, 2017, pp. 1–9.
Crossref Google Scholar
[34]
X. Zheng, Z. Cai, J. Yu, C. Wang, and Y. Li, Follow but no track: Privacy preserved profile publishing in cyber-physical social systems, IEEE Internet Things J., vol. 4, no. 6, pp. 1868–1878, 2017.
Crossref Google Scholar
[35]
X. Zheng, Z. Cai, and Y. Li, Data linkage in smart internet of things systems: A consideration from a privacy perspective, IEEE Commun. Mag., vol. 56, no. 9, pp. 55–61, 2018.
Crossref Google Scholar
[36]
B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, Y. Burren, N. Porz, J. Slotboom, R. Wiest, et al., The multimodal brain tumor image segmentation benchmark (BRATS), IEEE Trans. Med. Imaging, vol. 34, no. 10, pp. 1993–2024, 2015.
Crossref Google Scholar
[37]
F. Isersee, J. Petersen, A. Klein, D. Zimmerer, P. F. Jaeger, S, Kohl, J. Wasserthal, G. Koehler, T. Norajrtra, S. Wirkert, et al., nnU-Net: Self-adapting framework for U-Net-based modical image segmantation, arXiv preprint arXiv: 1809.10486, 2018.
Google Scholar
Tsinghua Science and Technology
Volume 28 Issue 4,
August 2023
Pages 628-638
DOI: 10.26599/TST.2022.9010023
Cite this article:
Zhang L, Zhang K, Pan H. SUNet++: A Deep Network with Channel Attention for Small-Scale Object Segmentation on 3D Medical Images. Tsinghua Science and Technology, 2023, 28(4): 628-638. https://doi.org/10.26599/TST.2022.9010023

1159

Views

248

Downloads

18

Crossref

11

Web of Science

19

Scopus

0

CSCD

Altmetrics
Received: 17 May 2022
Accepted: 21 June 2022
Published: 06 January 2023
© The author(s) 2023.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return