AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (3.2 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Lesson Learned from COVID-19 Retrospective Study: An Entropy-Based Clinical-Interpretable Scorecard for Mortality Risk Control at ICU Admission

Laboratory of Network Information Security, Beihang University, Beijing 100191, China
Andrew and Erna Viterbi School of Engineering, University of Southern California, Los Angles, CA 90089, USA
Beijing Goodwill Information and Technology Co., Ltd., and Mobile Health Laboratory, Peking University, Beijing 100871, China
Show Author Information

Abstract

With severe acute respiratory syndrome coronavirus 2 spreading globally and causing 2019 coronavirus disease (COVID-19), a challenge that we unprepared for was about how to optimally plan and distribute limited top-medical resources for patients in need of urgent care. To address this challenge, physicians desperately needed a scientific tool to methodically differentiate between cases with varying severity. In this study, the unique data of COVID-19 intensive care unit (ICU) patients provided by the national medical team in Wuhan were classified into discrete and continuous variable types. All continuous data were discretized using an entropy-based method and transformed into serial information margins, in which each information margin is related to a specific symptom or clinical meaning. Finally, all these native and processed discrete data were used to configure a readable scorecard through logistic regression, which is the desired scientific tool aforementioned. A total of 322 ICU patients (age: [median: 64, interquartile range: 54–75], males: 178 [55.28%], and death: 72 [22.36%]) were included in the study. Probabilities of mortality in COVID-19 patients can be evaluated using a scorecard model (calibration slope: 1.343, Brier: 0.048, Dxy = 0.972, and population stability index = 0.071), with desired model performances (accuracy = 0.948, area under curve = 0.99, sensitivity = 1, and specificity = 0.939). This new model can interpret clinical meanings from complex data, and compare it with existing machine learning methods through a black-box mechanism. This new data-information model answers a critical question of how a computing algorithm produces clinically meaningful results that will help physicians logically allocate medical resources for COVID-19 patients. Notably, this tool has limitations, giving that this research is a retrospective study. Hopefully, this tool will be tested further and optimized for adaptation to similar clinical cases in the future.

Keywords

COVID-19scorecardclinical-interpretablemachine learningICU admission control

References

[1]
M. Esai Selvan, Risk factors for death from COVID-19, Nat. Rev. Immunol., vol. 20, no. 7, p. 407, 2020.
Crossref Google Scholar
[2]
Y. Shang, T. Liu, Y. Wei, J. Li, L. Shao, M. Liu, Y. Zhang, Z. Zhao, H. Xu, Z. Peng, et al., Scoring systems for predicting mortality for severe patients with COVID-19, eClinicalMedicine, vol. 24, p. 100426, 2020.
Crossref Google Scholar
[3]
K. A. Overmyer, E. Shishkova, I. J. Miller, J. Balnis, M. N. Bernstein, T. M. Peters-Clarke, J. G. Meyer, Q. Quan, L. K. Muehlbauer, E. A. Trujillo, et al., Large-scale multi-omic analysis of COVID-19 severity, Clin. Transl. Discov., vol. 12, no. 1, pp. 23–40, 2021.
Crossref Google Scholar
[4]
G. Zhang, Y. An, L. Zhang, L. Xie, and X. Guo, Risk factors for in-hospital mortality in patients with cancer and COVID-19, Lancet Oncol., vol. 21, no. 9, p. 407, 2020.
Crossref Google Scholar
[5]
J. Tian, X. Yuan, J. Xiao, Q. Zhong, C. Yang, B. Liu, Y. Cai, Z. Lu, J. Wang, Y. Wang, et al., Clinical characteristics and risk factors associated with COVID-19 disease severity in patients with cancer in Wuhan, China: A multicentre, retrospective, cohort study, Lancet Oncol., vol. 21, no. 7, pp. 893–903, 2020.
Crossref Google Scholar
[6]
W. Liang, H. Liang, L. Ou, B. Chen, A. Chen, C. Li, Y. Li, W. Guan, L. Sang, J. Lu, et al., Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19, JAMA Intern. Med., vol. 180, no. 8, pp. 1081–1089, 2020.
Crossref Google Scholar
[7]
L. Yan, H. T. Zhang, J. Goncalves, Y. Xiao, M. Wang, Y. Guo, C. Sun, X. Tang, L. Jing, M. Zhang, et al., An interpretable mortality prediction model for COVID-19 patients, Nat. Mach. Intell., vol. 2, no. 5, pp. 283–288, 2020.
Crossref Google Scholar
[8]
Y. Gao, G. Y. Cai, W. Fang, H. Y. Li, S. Y. Wang, L. Chen, Y. Yu, D. Liu, S. Xu, P. F. Cui, et al., Machine learning based early warning system enables accurate mortality risk prediction for COVID-19, Nat. Commun., vol. 11, no. 1, p. 5033, 2020.
Crossref Google Scholar
[9]
A. S. Yadaw, Y. C. Li, S. Bose, R. Iyengar, S. Bunyavanich, and G. Pandey, Clinical features of COVID-19 mortality: Development and validation of a clinical prediction model, Lancet Digit. Heath., vol. 2, no. 10, pp. 516–525, 2020.
Crossref Google Scholar
[10]
S. R. Knight, A. Ho, R. Pius, I. Buchan, G. Carson, T. M. Drake, J. Dunning, C. J. Fairfield, C. Gamble, C. A. Green, et al., Risk stratification of patients admitted to hospital with COVID-19 using the ISARIC WHO Clinical Characterisation Protocol: Development and validation of the 4C Mortality Score, BMJ Clin. Res. Ed., vol. 370, p. 3339, 2020.
Google Scholar
[11]
N. Razavian, V. J. Major, M. Sudarshan, J. Burk-Rafel, P. Stella, H. Randhawa, S. Bilaloglu, J. Chen, V. Nguy, W. Wang, et al., A validated, real-time prediction model for favorable outcomes in hospitalized COVID-19 patients, NPJ Digit. Med., vol. 3, p. 130, 2020.
Crossref Google Scholar
[12]
O. Troyanskaya, M. Cantor, G. Sherlock, P. Brown, T. Hastie, R. Tibshirani, D. Botstein, and R. B. Altman, Missing value estimation methods for DNA microarrays, Bioinform. Oxf. Engl., vol. 17, no. 6, pp. 520–525, 2001.
Crossref Google Scholar
[13]
J. Gupta, S. Paul, and A. Ghosh, A novel transfer learning-based missing value imputation on discipline diverse real test datasets—A comparative study with different machine learning algorithms, in Advances in Intelligent Systems and Computing, Singapore: Springer, 2019.
Crossref Google Scholar
[14]
, Rajan Vohra, and , Missing value imputation in multi attribute data set, Int. J. Comput. Sci. Infor. Technol., vol. 5, no. 4, pp. 5315–5321, 2014.
Google Scholar
[15]
G. E. A. P. A. Batista and M. C. Monard, An analysis of four missing data treatment methods for supervised learning, Appl. Artif. Intell., vol. 17, nos. 5–6, pp. 519–533, 2003.
Crossref Google Scholar
[16]
J. Dougherty, R. Kohavi, and Sahami M., Supervised and unsupervised discretization of continuous features, in Proc. 12th Int. Conf. Machine Learning, Tahoe City, CA, USA: Morgan Kaufmann, 1995.
Crossref Google Scholar
[17]
G. Zeng, Metric divergence measures and information value in credit scoring, J. Math., vol. 2013, pp. 1–10, 2013.
Crossref Google Scholar
[18]
M. Refaat, Credit Risk Scorecards: Development and Implementation Using SAS, Raleigh, NC, USA: LULU.COM, 2011.
[19]
N. Siddiqi, Credit Risk Scorecards: Developing and Implementing Intelligent Credit Scoring, Hoboken, NJ, USA: John Wiley & Sons, Inc., 2012.
Crossref
[20]
E. W. Steyerberg, A. J. Vickers, N. R. Cook, T. Gerds, M. Gonen, N. Obuchowski, M. J. Pencina, and M. W. Kattan, Assessing the performance of prediction models: A framework for traditional and novel measures, Epidemiol. Camb. Mass, vol. 21, no. 1, pp. 128–138, 2010.
Crossref Google Scholar
[21]
R. Taplin and C. Hunt, The population accuracy index: A new measure of population stability for model monitoring, Risks, vol. 7, no. 2, p. 53, 2019.
Crossref Google Scholar
[22]
A. J. Vickers and E. B. Elkin, Decision curve analysis: A novel method for evaluating prediction models, medical decision making, Med. Decis. Making, vol. 26, no. 6, pp. 565–574, 2006.
Crossref Google Scholar
[23]
A. J. Vickers, B. van Calster, and E. W. Steyerberg, A simple, step-by-step guide to interpreting decision curve analysis, Diagn. Progn. Res., vol. 3, p. 18, 2019.
Crossref Google Scholar
[24]
M. Fitzgerald, B. R. Saville, and R. J. Lewis, Decision curve analysis, JAMA, vol. 313, no. 4, p. 409, 2015.
Crossref Google Scholar
[25]
H. He, Y. Bai, E. A. Garcia, and S. Li, ADASYN: Adaptive synthetic sampling approach for imbalanced learning, in Proc. 2008 IEEE Int. Joint Conf. Neural Networks (IEEE World Congress on Computational Intelligence), Hong Kong, China, 2008, pp. 1322–1328.
Google Scholar
[26]
F. Caramelo, N. Ferreira, and B. Oliveiros, Estimation of risk factors for COVID-19 mortality-preliminary results, medRxiv, https://europepmc.org/article/PPR/PPR114369, 2020.
Crossref
[27]
B. Zheng, Y. Cai, F. Zeng, M. Lin, J. Zheng, W. Chen, G. Qin, and Y. Guo, An interpretable model-based prediction of severity and crucial factors in patients with COVID-19, BioMed Res. Int., vol. 2021, pp. 1–9, 2021.
Crossref Google Scholar
[28]
J. A. Kline, C. A. Camargo, D. M. Courtney, C. Kabrhel, K. E. Nordenholz, T. Aufderbeide, J. J. Baugh, D. G. Beiser, C. L. Bennett, J. Bledsoe, et al., Clinical prediction rule for SARS-CoV-2 infection from 116 U.S. emergency departments 2-22-2021, PloS One, vol. 16, no. 3, p. 0248438, 2021.
Crossref Google Scholar
[29]
K. B. Son, T. J. Lee, and S. S. Hwang, Disease severity classification and COVID-19 outcomes, Republic of Korea, Bull. World Heath. Organ., vol. 99, no. 1, pp. 62–66, 2021.
Crossref Google Scholar
[30]
M. Laforge, C. Elbim, C. Frère, M. Hémadi, C. Massaad, P. Nuss, J. J. Benoliel, and C. Becker, Tissue damage from neutrophil-induced oxidative stress in COVID-19, Nat. Rev. Immunol., vol. 20, no. 9, pp. 515–516, 2020.
Crossref Google Scholar
[31]
B. Kalyanaraman, Do free radical network and oxidative stress disparities in African Americans enhance their vulnerability to SARS-CoV-2 infection and COVID-19 severity? Redox Biol., vol. 37, p. 101721, 2020.
Crossref Google Scholar
[32]
P. Pan, Y. Li, Y. Xiao, B. Han, L. Su, M. Su, Y. Li, S. Zhang, D. Jiang, X. Chen, et al., Prognostic assessment of COVID-19 in the intensive care unit by machine learning methods: Model development and validation, J. Med. Internet Res., vol. 22, no. 11, p. 23128, 2020.
Crossref Google Scholar
[33]
A. Alnor, M. B. Sandberg, C. Gils, and P. J. Vinholt, Laboratory tests and outcome for patients with coronavirus disease 2019: A systematic review and meta-analysis, J. Appl. Lab. Med., vol. 5, no. 5, pp. 1038–1049, 2020.
Crossref Google Scholar
[34]
X. Zhang, Y. Tan, Y. Ling, G. Lu, F. Liu, Z. Yi, X. Jia, M. Wu, B. Shi, S. Xu, et al., Viral and host factors related to the clinical outcome of COVID-19, Nature, vol. 583, no. 7816, pp. 437–440, 2020.
Crossref Google Scholar
[35]
D. Wang, B. Hu, C. Hu, F. Zhu, X. Liu, J. Zhang, B. Wang, H. Xiang, Z. Cheng, Y. Xiong, et al., Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus–infected pneumonia in Wuhan, China, JAMA, vol. 323, no. 11, p. 1061, 2020.
Crossref Google Scholar
[36]
F. He, Y. Quan, M. Lei, R. Liu, S. Qin, J. Zeng, Z. Zhao, N. Yu, L. Yang, and J. Cao, Clinical features and risk factors for ICU admission in COVID-19 patients with cardiovascular diseases, Aging Dis., vol. 11, no. 4, p. 763, 2020.
Crossref Google Scholar
[37]
W. S. Lim, M. M. van der Eerden, R. Laing, W. G. Boersma, N. Karalus, G. I. Town, S. A. Lewis, and J. T. MacFarlane, Defining community acquired pneumonia severity on presentation to hospital: An international derivation and validation study, Thorax, vol. 58, no. 5, pp. 377–382, 2003.
Crossref Google Scholar
[38]
M. J. Fine, T. E. Auble, D. M. Yealy, B. H. Hanusa, L. A. Weissfeld, D. E. Singer, C. M. Coley, T. J. Marrie, and W. N. Kapoor, A prediction rule to identify low-risk patients with community-acquired pneumonia, Dev. Camb. Engl., vol. 336, no. 4, pp. 243–250, 1997.
Crossref Google Scholar
[39]
J. L. Liu, F. Xu, H. Zhou, X. J. Wu, L. X. Shi, R. Q. Lu, A. Farcomeni, M. Venditti, Y. L. Zhao, S. Y. Luo, et al., Expanded CURB-65: A new score system predicts severity of community-acquired pneumonia with superior efficiency, Sci. Rep., vol. 6, p. 22911, 2016.
Crossref Google Scholar
Tsinghua Science and Technology
Volume 29 Issue 1,
February 2024
Pages 34-45
DOI: 10.26599/TST.2023.9010042
Cite this article:
Yao C, Huangqi C, Huang A. Lesson Learned from COVID-19 Retrospective Study: An Entropy-Based Clinical-Interpretable Scorecard for Mortality Risk Control at ICU Admission. Tsinghua Science and Technology, 2024, 29(1): 34-45. https://doi.org/10.26599/TST.2023.9010042

624

Views

21

Downloads

0

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 27 November 2022
Revised: 27 March 2023
Accepted: 09 May 2023
Published: 21 August 2023
© The author(s) 2024.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return