AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Article Link
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Regular Paper

Robust Needle Localization and Enhancement Algorithm for Ultrasound by Deep Learning and Beam Steering Methods

College of Computer Science, Sichuan University, Chengdu 610065, China
Stork Healthcare, Chengdu 610000, China
College of Computer and Information Science, Southwest University, Chongqing 400715, China
West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu 610065, China
Pera Corporation Ltd., Beijing 100025, China
Show Author Information

Abstract

Ultrasound (US) imaging is clinically used to guide needle insertions because it is safe, real-time, and low cost. The localization of the needle in the ultrasound image, however, remains a challenging problem due to specular reflection off the smooth surface of the needle, speckle noise, and similar line-like anatomical features. This study presents a novel robust needle localization and enhancement algorithm based on deep learning and beam steering methods with three key innovations. First, we employ beam steering to maximize the reflection intensity of the needle, which can help us to detect and locate the needle precisely. Second, we modify the U-Net which is an end-to-end network commonly used in biomedical segmentation by using two branches instead of one in the last up-sampling layer and adding three layers after the last down-sample layer. Thus, the modified U-Net can real-time segment the needle shaft region, detect the needle tip landmark location and determine whether an image frame contains the needle by one shot. Third, we develop a needle fusion framework that employs the outputs of the multi-task deep learning (MTL) framework to precisely locate the needle tip and enhance needle shaft visualization. Thus, the proposed algorithm can not only greatly reduce the processing time, but also significantly increase the needle localization accuracy and enhance the needle visualization for real-time clinical intervention applications.

Electronic Supplementary Material

Download File(s)
jcst-36-2-334-Highlights.pdf (155.3 KB)

References

[1]

Gulsen F, Kantarci F. Ultrasound-guided intervention around the hip joint. American Journal of Roentgenology, 2012, 198(1): W95. https://doi.org/10.2214/AJR.11.7627.

[2]

Chin K J, Perlas A, Chan V W S, Brull R. Needle visualization in ultrasound-guided regional anesthesia: Challenges and solutions. Regional Anesthesia and Pain Medicine, 2008, 33(6): 532-544. https://doi.org/10.1016/j.rapm.2008.06.002.

[3]

Fevre M C, Vincent C, Picard J, Vighetti A, Chapuis C, Detavernier M, Allenet B, Payen J F, Bosson J L, Albaladejo P. Reduced variability and execution time to reach a target with a needle GPS system: Comparison between physicians, residents and nurse anaesthetists. Anaesthesia Critical Care & Pain Medicine, 2018, 37(1): 55-60. https://doi.org/10.1016/j.accpm.2016.05.008.

[4]
Stolka P J, Foroughi P, Rendina M, Weiss C R, Hager G D, Boctor E M. Needle guidance using handheld stereo vision and projection for ultrasound-based interventions. In Proc. the 17th International Conference on Medical Image Computing and Computer-Assisted Intervention, September 2014, pp.684-691. https://doi.org/10.1007/978-3-319-10470-6_85.
[5]
Lu H, Li J, Lu Q, Bharat S, Erkamp R, Chen B, Drysdale J, Vignon F, Jain A. A new sensor technology for 2D ultrasound-guided needle tracking. In Proc. the 17th International Conference on Medical Image Computing and Computer-Assisted Intervention, September 2014, pp.389-396. https://doi.org/10.1007/978-3-319-10470-6_49.
[6]

Xia W, West S J, Finlay M C, Mari J M, Ourselin S, David A L, Desjardins A E. Looking beyond the imaging plane: 3D needle tracking with a linear array ultrasound probe. Sci. Rep., 2017, 7(1): Article No. 3674. https://doi.org/10.1038/s41598-017-03886-4.

[7]
Ding M, Fenster A. Projection-based needle segmentation in 3D ultrasound images. In Proc. the 6th International Conference on Medical Image Computing and Computer-Assisted Intervention, November 2003, pp.319-327. https://doi.org/10.3109/10929080500079321.
[8]

Qiu W, Yuchi M, Ding M. Phase grouping-based needle segmentation in 3-D trans-rectal ultrasound-guided prostate trans-perineal therapy. Ultrasound in Medicine & Biology, 2014, 40(4): 804-816. https://doi.org/10.1016/j.ultrasmedbio.2013.11.004.

[9]

Beigi P, Rohling R, Salcudean T, Lessoway V A, Ng G C. Needle trajectory and tip localization in real-time 3-D ultrasound using a moving stylus. Ultrasound in Medicine & Biology, 2015, 41(7): 2057-2070. https://doi.org/10.1016/j.ultrasmedbio.2015.03.013.

[10]

Zhao Y, Shen Y, Bernard A, Cachard C, Liebgott H. Evaluation and comparison of current biopsy needle localization and tracking methods using 3D ultrasound. Ultrasonics, 2017, 73: 206-220. https://doi.org/10.1016/j.ultras.2016.09.006.

[11]

Ayvali E, Desai J P. Optical ow-based tracking of needles and needle-tip localization using circular hough transform in ultrasound images. Annals of Biomedical Engineering, 2015, 43(8): 1828-1840. https://doi.org/10.1007/s10439-014-1208-0.

[12]

Uhercik M, Kybic J, Liebgott H, Cachard C. Model fitting using RANSAC for surgical tool localization in 3D ultrasound images. IEEE Transactions on Biomedical Engineering, 2010, 57(8): 1907-1916. https://doi.org/10.1109/TBME.2010.2046416.

[13]

Zhao Y, Cachard C, Liebgott H. Automatic needle detection and tracking in 3D ultrasound using an ROI-based RANSAC and Kalman method. Ultrasonic Imaging, 2013, 35(4): 283-306. https://doi.org/10.1177/0161734613502004.

[14]

Mwikirize C, Nosher J L, Hacihaliloglu I. Learning needle tip localization from digital subtraction in 2D ultrasound. International Journal of Computer Assisted Radiology and Surgery, 2019, 14(6): 1017-1026. https://doi.org/10.1007/s11548-019-01951-z.

[15]

Mwikirize C, Nosher J L, Hacihaliloglu I. Convolution neural networks for real-time needle detection and localization in 2D ultrasound. International Journal of Computer Assisted Radiology and Surgery, 2018, 13(5): 647-657. https://doi.org/10.1007/s11548-018-1721-y.

[16]

Beigi P, Rohling R, Salcudean S E, Ng G C. CASPER: Computer-aided segmentation of imperceptible motion—A learning-based tracking of an invisible needle in ultrasound. International Journal of Computer Assisted Radiology and Surgery, 2017, 12(11): 1857-1866. https://doi.org/10.1007/s11548-017-1631-4.

[17]

Pourtaherian A, Ghazvinian Zanjani F, Zinger S, Mihajlovic N, Ng G C, Korsten H H M, De With P H N. Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks. International Journal of Computer Assisted Radiology and Surgery, 2018, 13(9): 1321-1333. https://doi.org/10.1007/s11548-018-1798-3.

[18]

Arif M, Moelker A, Van Walsum T. Automatic needle detection and real-time Bi-planar needle visualization during 3D ultrasound scanning of the liver. Medical Image Analysis, 2019, 53: 104-110. https://doi.org/10.1016/j.media.2019.02.002.

[19]
Mwikirize C, Nosher J L, Hacihaliloglu I. Single shot needle tip localization in 2D ultrasound. In Proc. the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, October 2019, pp.637-645. https://doi.org/10.1007/978-3-030-32254-0_71.
[20]

Hatt C R, Ng G, Parthasarathy V. Enhanced needle localization in ultrasound using beam steering and learning-based segmentation. Computerized Medical Imaging and Graphics, 2015, 41: 46-54. https://doi.org/10.1016/j.compmedimag.2014.06.016.

[21]
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In Proc. the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, October 2015, pp. 234-241. https://doi.org/10.1007/978-3-319-24574-4_28.
[22]
Gao J, Liu P, Liu G D, Zhang L. Supplementary material. Technical Report, Sichuan University, 2020. https://github.com/gaojun0821/NLEM/blob/main/doc/-Supplementary_Material.pdf, Dec. 2020.
[23]

Jiang B, Struthers A, Sun Z, Feng Z, Zhao X, Zhao K, Dai W, Zhou X, Berens M E, Zhang L. Employing graphics processing unit technology, alternating direction implicit method and domain decomposition to speed up the numerical diffusion solver for the biomedical engineering research. International Journal for Numerical Methods in Biomedical Engineering, 2011, 27(11): 1829-1849. https://doi.org/10.1002/cnm.1444.

[24]

Jiang B N, Dai W Z, Khaliq A, Carey M, Zhou X B, Zhang L. Novel 3D GPU based numerical parallel diffusion algorithms in cylindrical coordinates for health care simulation. Mathematics and Computers in Simulation, 2015, 109: 1-19. https://doi.org/10.1016/j.matcom.2014.07.003.

[25]

Zhang L, Jiang B, Wu Y, Strouthos C, Sun P Z, Su J, Zhou X. Developing a multiscale, multi-resolution agent-based brain tumor model by graphics processing units. Theoretical Biology and Medical Modelling, 2011, 8(1): Article No. 46. https://doi.org/10.1186/1742-4682-8-46.

[26]
Isola P, Zhu J Y, Zhou T, Efros A. Image-to-image translation with conditional adversarial networks. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, July 2017, pp.5967-5976. https://doi.org/10.1109/CVPR.2017.632.
[27]

Wu W, Song L, Yang Y, Wang J, Liu H, Zhang L. Exploring the dynamics and interplay of human papillomavirus and cervical tumorigenesis by integrating biological data into a mathematical model. BMC Bioinformatics, 2020, 21(Suppl 7): Article No. 152. https://doi.org/10.1186/s12859-020-3454-5.

[28]

Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. https://doi.org/10.1109/TPAMI.2016.2577031.

[29]
Xu Z, Huo Y, Park J, Landman B, Milkowski A, Grbic S, Zhou S. Less is more: Simultaneous view classification and landmark detection for abdominal ultrasound images. In Proc. the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, September 2018, pp.711-719. https://doi.org/10.1007/978-3-030-00934-2_79.
[30]
Powers D. Evaluation: From precision, recall and F-factor to ROC, informedness, markedness & correlation. arXiv: 2010.16061, 2008. https://arxiv.org/abs/2010.16061, Jan. 2021.
[31]

Xia Y, Yang C, Hu N, Yang Z, He X, Li T, Zhang L. Exploring the key genes and signaling transduction pathways related to the survival time of glioblastoma multiforme patients by a novel survival analysis model. BMC Genomics, 2017, 18(Suppl 1): Article No. 950. https://doi.org/10.1186/s12864-016-3256-3.

[32]

Zhang L, Bai W, Yuan N, Du Z. Comprehensively bench-marking applications for detecting copy number variation. PLoS Computational Biology, 2019, 15(5): Article No. e1007069. https://doi.org/10.1371/journal.pcbi.1007069.

[33]

Zhang L, Li J, Yin K, Jiang Z, Li T, Hu R, Yu Z, Feng H, Chen Y. Computed tomography angiography-based analysis of high-risk intracerebral haemorrhage patients by employing a mathematical model. BMC Bioinformatics, 2019, 20(Suppl 7): Article No. 193. https://doi.org/10.1186/s12859-019-2741-5.

[34]

Zhang L, Liu Y, Wang M, Wu Z, Li N, Zhang J, Yang C. EZH2-, CHD4-, and IDH-linked epigenetic perturbation and its association with survival in glioma patients. J. Mol. Cell Biol., 2017, 9(6): 477-488. https://doi.org/10.1093/jmcb/mjx056.

[35]

Kremkau F W, Taylor K J. Artifacts in ultrasound imaging. Journal of Ultrasound in Medicine, 1986, 5(4): 227-237. https://doi.org/10.7863/jum.1986.5.4.227.

[36]

Paul Y, Barthez D, Léveillé R, Peter V, Scrivani D. Side lobes and grating lobes artifacts in ultrasound imaging. Veterinary Radiology & Ultrasound, 1997, 38(5): 387-393. https://doi.org/10.1111/j.1740-8261.1997.tb02104.x.

[37]

Matalon T A, Silver B. US guidance of interventional procedures. Radiology, 1990, 174(1): 43-47. https://doi.org/10.1148/radiology.174.1.2403684.

[38]

Sofka C, Collins A J, Adler R. Use of ultrasonographic guidance in interventional musculoskeletal procedures: A review from a single institution. Journal of Ultrasound in Medicine: Official Journal of the American Institute of Ultrasound in Medicine, 2001, 20(1): 21-26. https://doi.org/10.7863/jum.2001.20.1.21.

Journal of Computer Science and Technology
Pages 334-346
Cite this article:
Gao J, Liu P, Liu G-D, et al. Robust Needle Localization and Enhancement Algorithm for Ultrasound by Deep Learning and Beam Steering Methods. Journal of Computer Science and Technology, 2021, 36(2): 334-346. https://doi.org/10.1007/s11390-021-0861-7

433

Views

34

Crossref

28

Web of Science

33

Scopus

4

CSCD

Altmetrics

Received: 29 July 2020
Accepted: 25 February 2021
Published: 05 March 2021
©Institute of Computing Technology, Chinese Academy of Sciences 2021
Return