AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (2.8 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Parallel ADR Detection Based on Spark and BCPNN

Li Sun, Shan Sun, Tianlei Wang, Jiyun Li( ), Jingsheng Lin
School of Computer Science and Technology, Donghua University, Shanghai 201620, China.
Ruijin Hospital Affiliated to Shanghai Jiao Tong University, Shanghai 200020, China.
Show Author Information

Abstract

Adverse Drug Reaction (ADR) is one of the major challenges to the evaluation of drug safety in the medical field. The Bayesian Confidence Propagation Neural Network (BCPNN) algorithm is the main algorithm used by the World Health Organization to monitor ADRs. Currently, ADR reports are collected through the spontaneous reporting system. However, with the continuous increase in ADR reports and possible use scenarios, the efficiency of the stand-alone ADR detection algorithm will encounter considerable challenges. Meanwhile, the BCPNN algorithm requires a certain number of disk I/O, which leads to considerable time consumption. In this study, we propose a Spark-based parallel BCPNN algorithm, which speeds up data processing and reduces the number of disk I/O in BCPNN, and two optimization strategies. Then, the ADR data collected from the FDA Adverse Event Reporting System are used to verify the performance of the proposed algorithm and its optimization strategies. Experiments show that the parallel BCPNN can significantly accelerate data processing and the optimized algorithm has a high acceleration rate and can effectively prevent memory overflow. Finally, we apply the proposed algorithm to a dataset provided by a real medical consortium. Experiments further prove the performance and practical value of the proposed algorithm.

Keywords

Adverse Drug Reaction (ADR)Bayesian Confidence Propagation Neural Network (BCPNN)parallelSpark

References

[1]
Bate A., Lindquist M., Edwards I. R., and Orre R., A data mining approach for signal detection and analysis, Drug Saf., vol. 25, no. 6, pp. 393-397, 2002.
Crossref Google Scholar
[2]
Bate A., The use of a Bayesian confidence propagation neural network in pharmacovigilance, PhD dissertation, Umeå University, Sweden, 2003.
[3]
Karimi S., Wang C., Metke-Jimenez A., Gaire R., and Paris C., Text and data mining techniques in adverse drug reaction detection, ACM Computing Surveys, vol. 47, no. 4, p. 56, 2015.
Crossref Google Scholar
[4]
Chen W. G. and Deng J. X., A study on signal detection and automatic warning algorithm for adverse drug reaction, in Proc. 2008 International Conference on Computer Science and Software Engineering, 2008.
[5]
Swetha K. V., Sathyadevan S., and Bilna P., Network data analysis using spark, in Software Engineering in Intelligent Systems, Silhavy R., Senkerik R., Oplatkova Z., Prokopova Z., and Silhavy P., eds. Springer, 2015, pp. 253-259.
Crossref
[6]
Bate A., Lindquist M., Edwards I. R., Olsson S., Orre R., Lansner A., and De Freitas R. M., A Bayesian neural network method for adverse drug reaction signal generation, Eur. J. Clin. Pharmacol., vol. 54, no. 4, pp. 315-321, 1998.
Crossref Google Scholar
[7]
Farahini N., Hemani A., Lansner A., Clermidy F., and Svensson C., A scalable custom simulation machine for the Bayesian Confidence Propagation Neural Network model of the brain. in Proc. 2014 19th Asia and South Pacific Design Automation Conf., Singapore, 2014, pp. 578-585.
Crossref
[8]
Honigman B., Lee J., Rothschild J., Light P., Pulling R. M., Yu T., and Bates D. W., Using computerized data to identify adverse drug events in outpatients, J. Am. Med. Inform. Assoc., vol. 8, no. 3, pp. 254-266, 2001.
Crossref Google Scholar
[9]
Duan L., Khoshneshin M., Street W. N., and Liu M., Adverse drug effect detection, IEEE Journal of Biomedical & Health Informatics, vol. 17, no. 2, pp. 305-311, 2013.
Crossref Google Scholar
[10]
Bates D. W., Evans R. S., Murff H., Stetson P. D., Pizziferri L., and Hripcsak G., Detecting adverse events using information technology, Journal of the American Medical Informatics Association, vol. 10, no. 2, pp. 115-128, 2003.
Crossref Google Scholar
[11]
Han Z. J. and Zhang Y. J., Spark: A big data processing platform based on memory computing, in Proc. 7th Int. Symposium on Parallel Architectures, Algorithms and Programming, Nanjing, China, 2015, pp. 172-176.
Crossref
[12]
Solovyev A., Mikheev M., Zhou L. M., Dutta-Moscato J., Ziraldo C., An G., Vodovotz Y., and Mi Q., SPARK: A framework for multi-scale agent-based biomedical modeling, International Journal of Agent Technologies & Systems, vol. 2, no. 3, pp. 18-30, 2010.
Crossref Google Scholar
[13]
Cao P., Optimization and implementation of clustering algorithm based on Spark platform, (in Chinese), master degree dissertation, Beijing Jiaotong University, Beijing, China, 2016.
[14]
Zhang A., Insider of Spark Technology, (in Chinese). Beijing, China: Mechanical Industry Press, 2015.
[15]
Fu J., Sun J. W., and Wang K. Y., SPARK—A big data processing platform for machine learning, in Proc. 2016 International Conference on Industrial Informatics - Computing Technology, Intelligent Technology, Industrial Information Integration, Wuhan, China, 2017, pp. 48-51.
Crossref
[16]
Brewer T. and Colditz G. A., Postmarketing surveillance and adverse drug reactions: Current perspectives and future needs, JAMA, vol. 218, no. 9, pp. 824-829, 1999.
Crossref Google Scholar
[17]
Lansner A. and Ekeberg Ö., A one-layer feedback artificial neural network with a Bayesian learning rule, International Journal of Neural Systems, vol. 1, no. 1, pp. 77-87, 1989.
Crossref Google Scholar
[18]
Karau H., Konwinski A., Wendell P., and Zaharia M., Learning Spark: Lightning-Fast Data Analysis, (in Chinese). Beijing, China: People’s Posts and Telecommunications Press, 2015.
[19]
Li W. Y., Research on apache spark for big data processing, (in Chinese), Modern Computer, no. 8, pp. 55-60, 2015.
Google Scholar
[20]
Xie S. L., Research and application of distributed ETL based on spark, (in Chinese), master degree dissertation, Donghua University, Shanghai, China, 2017.
[21]
FDA adverse event reporting system (FAERS): Latest quarterly data files, https://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugEffects/ucm082193.htm, 2017.
[22]
Wu Z. F., Zhang T., and Xiao Y., Improvement and parallel implementation of K-means clustering algorithm based on the Spark platform, (in Chinese), China Internet, no. 1, pp. 44-50, 2016.
Google Scholar
[23]
Wang P. F. and Li L., Research on multi-pattern matching algorithms based on Aho-Corasick algorithm, (in Chinese), Application Research of Computers, vol. 28, no. 4, pp. 1251–1253&1259, 2011.
Google Scholar
[24]
Aho A. V. and Corasick M. J., Efficient string matching: An aid to bibliographic search, Communications of the ACM, vol. 18, no. 6, pp. 333-340, 1975.
Crossref Google Scholar
Tsinghua Science and Technology
Volume 24 Issue 2,
April 2019
Pages 195-206
DOI: 10.26599/TST.2018.9010074
Cite this article:
Sun L, Sun S, Wang T, et al. Parallel ADR Detection Based on Spark and BCPNN. Tsinghua Science and Technology, 2019, 24(2): 195-206. https://doi.org/10.26599/TST.2018.9010074

714

Views

60

Downloads

6

Crossref

N/A

Web of Science

12

Scopus

1

CSCD

Altmetrics
Received: 16 August 2017
Accepted: 10 January 2018
Published: 31 December 2018
© The author(s) 2019
Return