AI Chat Paper
Discover the SciOpen Platform and Achieve Your Research Goals with Ease.
Search articles, authors, keywords, DOl and etc.
{{expandStatus?'Exit ':''}}Advanced Search
Direct volume rendering (DVR) is a technique that emphasizes structures of interest (SOIs) within a volume visually, while simultaneously depicting adjacent regional information, e.g., the spatial location of a structure concerning its neighbors. In DVR, transfer function (TF) plays a key role by enabling accurate identification of SOIs interactively as well as ensuring appropriate visibility of them. TF generation typically involves non-intuitive trial-and-error optimization of rendering parameters, which is time-consuming and inefficient. Attempts at mitigating this manual process have led to approaches that make use of a knowledge database consisting of pre-designed TFs by domain experts. In these approaches, a user navigates the knowledge database to find the most suitable pre-designed TF for their input volume to visualize the SOIs. Although these approaches potentially reduce the workload to generate the TFs, they, however, require manual TF navigation of the knowledge database, as well as the likely fine tuning of the selected TF to suit the input. In this work, we propose a TF design approach, CBR-TF, where we introduce a new content-based retrieval (CBR) method to automatically navigate the knowledge database. Instead of pre-designed TFs, our knowledge database contains volumes with SOI labels. Given an input volume, our CBR-TF approach retrieves relevant volumes (with SOI labels) from the knowledge database; the retrieved labels are then used to generate and optimize TFs of the input. This approach largely reduces manual TF navigation and fine tuning. For our CBR-TF approach, we introduce a novel volumetric image feature which includes both a local primitive intensity profile along the SOIs and regional spatial semantics available from the co-planar images to the profile. For the regional spatial semantics, we adopt a convolutional neural network to obtain high-level image feature representations. For the intensity profile, we extend the dynamic time warping technique to address subtle alignment differences between similar profiles (SOIs). Finally, we propose a two-stage CBR scheme to enable the use of these two different feature representations in a complementary manner, thereby improving SOI retrieval performance. We demonstrate the capabilities of our CBR-TF approach with comparison with a conventional approach in visualization, where an intensity profile matching algorithm is used, and also with potential use-cases in medical volume visualization.
Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, Fillion-Robin J C, Pujol S, Bauer C, Jennings D, Fennessy F, Sonka M, Buatti J, Aylward S, Miller J V, Pieper S, Kikinis R. 3D Slicer as an image computing platform for the quantitative imaging network. Magnetic Resonance Imaging, 2012, 30(9): 1323–1341. DOI: 10.1016/j.mri.2012.05.001.
Ljung P, Krüger J, Groller E, Hadwiger M, Hansen C D, Ynnerman A. State of the art in transfer functions for direct volume rendering. Computer Graphics Forum, 2016, 35(3): 669–691. DOI: 10.1111/cgf.12934.
Correa C, Ma K L. Size-based transfer functions: A new volume exploration technique. IEEE Trans. Visualization and Computer Graphics, 2008, 14(6): 1380–1387. DOI: 10.1109/TVCG.2008.162.
Caban J J, Rheingans P. Texture-based transfer functions for direct volume rendering. IEEE Trans. Visualization and Computer Graphics, 2008, 14(6): 1364–1371. DOI: 10.1109/TVCG.2008.169.
Jung Y, Kim J, Kumar A, Feng D D, Fulham M. Feature of interest-based direct volume rendering using contextual saliency-driven ray profile analysis. Computer Graphics Forum, 2018, 37(6): 5–19. DOI: 10.1111/cgf.13308.
Guo H Q, Mao N Y, Yuan X R. WYSIWYG (what you see is what you get) volume visualization. IEEE Trans. Visualization and Computer Graphics, 2011, 17(12): 2106–2114. DOI: 10.1109/TVCG.2011.261.
Correa C D, Ma K L. Visibility histograms and visibility-driven transfer functions. IEEE Trans. Visualization and Computer Graphics, 2011, 17(2): 192–204. DOI: 10.1109/TVCG.2010.35.
Jung Y, Kim J, Eberl S, Fulham M, Feng D D. Visibility-driven PET-CT visualisation with region of interest (ROI) segmentation. The Visual Computer, 2013, 29(6): 805–815. DOI: 10.1007/s00371-013-0833-1.
Jung Y, Kim J, Kumar A, Feng D D, Fulham M. Efficient visibility-driven medical image visualisation via adaptive binned visibility histogram. Computerized Medical Imaging and Graphics, 2016, 51: 40–49. DOI: 10.1016/j.compmedimag.2016.04.003.
Jung Y, Kim J, Bi L, Kumar A, Feng D D, Fulham M. A direct volume rendering visualization approach for serial PET–CT scans that preserves anatomical consistency. International Journal of Computer Assisted Radiology and Surgery, 2019, 14(5): 733–744. DOI: 10.1007/s11548-019-01916-2.
LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521(7553): 436–444. DOI: 10.1038/nature14539.
Kumar A, Kim J, Cai W D, Fulham M, Feng D G. Content-based medical image retrieval: A survey of applications to multidimensional and multimodality data. Journal of Digital Imaging, 2013, 26(6): 1025–1039. DOI: 10.1007/s10278-013-9619-2.
Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60(6): 84–90. DOI: 10.1145/3065386.
Keogh E, Ratanamahatana C A. Exact indexing of dynamic time warping. Knowledge and Information Systems, 2005, 7(3): 358–386. DOI: 10.1007/s10115-004-0154-9.
Nelder J A, Mead R. A simplex method for function minimization. The Computer Journal, 1965, 7(4): 308–313. DOI: 10.1093/comjnl/7.4.308.
Lagarias J C, Reeds J A, Wright M H, Wright P E. Convergence properties of the Nelder-Mead simplex method in low dimensions. SIAM Journal on Optimization, 1998, 9(1): 112–147. DOI: 10.1137/S1052623496303470.
Shin H C, Roth H R, Gao M C, Lu L, Xu Z Y, Nogues I, Yao J H, Mollura D, Summers R M. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Medical Imaging, 2016, 35(5): 1285–1298. DOI: 10.1109/TMI.2016.2528162.
Meyer-Spradow J, Ropinski T, Mensmann J, Hinrichs K. Voreen: A rapid-prototyping environment for ray-casting-based volume visualizations. IEEE Computer Graphics and Applications, 2009, 29(6): 6–13. DOI: 10.1109/MCG.2009.130.
Ahmed K T, Ummesafi S, Iqbal A. Content based image retrieval using image features information fusion. Information Fusion, 2019, 51: 76–99. DOI: 10.1016/j.inffus.2018.11.004.
Vishraj R, Gupta S, Singh S. A comprehensive review of content-based image retrieval systems using deep learning and hand-crafted features in medical imaging: Research challenges and future directions. Computers and Electrical Engineering, 2022, 104: 108450. DOI: 10.1016/j.compeleceng.2022.108450.
Wasserthal J, Breit H C, Meyer M T, Pradella M, Hinck D, Sauter A W, Heye T, Boll D T, Cyriac J, Yang S, Bach M, Segeroth M. TotalSegmentator: Robust segmentation of 104 anatomic structures in CT images. Radiology: Artificial Intelligence, 2023, 5(5): e230024. DOI: 10.1148/ryai.230024.
Zhang C Y, Zheng H, Gu Y. Dive into the details of selfsupervised learning for medical image analysis. Medical Image Analysis, 2023, 89: 102879. DOI: 10.1016/j.media.2023.102879.
Chen X X, Wang X M, Zhang K, Fung K M, Thai T C, Moore K, Mannel R S, Liu H, Zheng B, Qiu Y C. Recent advances and clinical applications of deep learning in medical image analysis. Medical Image Analysis, 2022, 79: 102444. DOI: 10.1016/j.media.2022.102444.
Wang L, Qian X M, Zhang Y T, Shen J L, Cao X C. Enhancing sketch-based image retrieval by CNN semantic re-ranking. IEEE Trans. Cybernetics, 2020, 50(7): 3330–3342. DOI: 10.1109/TCYB.2019.2894498.
京公网安备11010802044758号