AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Article Link
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Survey

Machine Learning Techniques for Software Maintainability Prediction: Accuracy Analysis

Software Project Management Team, École Nationale Supérieure d’Informatique et d’Analyse des Systémes, Madinate Al Irfane, Mohammed V University in Rabat, Agdal Rabat, BP 713, Morocco
Department of Software Engineering & Information Technology, Ecole de Technologie Supérieure (ETS), Montréal H3C IK3, Canada
Show Author Information

Abstract

Maintaining software once implemented on the end-user side is laborious and, over its lifetime, is most often considerably more expensive than the initial software development. The prediction of software maintainability has emerged as an important research topic to address industry expectations for reducing costs, in particular, maintenance costs. Researchers and practitioners have been working on proposing and identifying a variety of techniques ranging from statistical to machine learning (ML) for better prediction of software maintainability. This review has been carried out to analyze the empirical evidence on the accuracy of software product maintainability prediction (SPMP) using ML techniques. This paper analyzes and discusses the findings of 77 selected studies published from 2000 to 2018 according to the following criteria: maintainability prediction techniques, validation methods, accuracy criteria, overall accuracy of ML techniques, and the techniques offering the best performance. The review process followed the well-known systematic review process. The results show that ML techniques are frequently used in predicting maintainability. In particular, artificial neural network (ANN), support vector machine/regression (SVM/R), regression & decision trees (DT), and fuzzy & neuro fuzzy (FNF) techniques are more accurate in terms of PRED and MMRE. The N-fold and leave-one-out cross-validation methods, and the MMRE and PRED accuracy criteria are frequently used in empirical studies. In general, ML techniques outperformed non-machine learning techniques, e.g., regression analysis (RA) techniques, while FNF outperformed SVM/R, DT, and ANN in most experiments. However, while many techniques were reported superior, no specific one can be identified as the best.

References

[1]

Abran A, Nguyenkim H. Measurement of the maintenance process from a demand-based perspective. J. Softw. Maint. Res. Pract., 1993, 5(2): 63-90.

[2]

Abran A, Bourque P, Dupuis R, Moore J W. Guide to the Software Engineering Body of Knowledge — SWEBOK. IEEE Press, 2001.

[3]
Riaz M, Mendes E, Tempero E. A systematic review of software maintainability prediction and metrics. In Proc. the 3rd International Symposium on Empirical Software Engineering and Measurement, October 2009, pp.367-377.
[4]
Riaz M. Maintainability prediction of relational databasedriven applications: A systematic review. In Proc. the 16th International Conference on Evaluation & Assessment in Software Engineering, May 2012, pp.263-272.
[5]
Orenyi B A. Basri S, Jung L T. Object-oriented software maintainability measurement in the past decade. In Proc. the 2012 International Conference on Advanced Computer Science Applications and Technologies, Nov. 2012, pp.257-262.
[6]
Dubey S K. Sharma A, Rana A. Analysis of maintainability models for object oriented system. Int. J. Comput. Sci. Eng., 2011, 3(12): 3837-3844.
[7]
Burrows R, Garcia A, Taïani F. Coupling metrics for aspectoriented programming: A systematic review of maintainability studies. In Proc. the 4th International Conference on Evaluation of Novel Approaches to Software Engineering, May 2009, pp.277-290.
[8]
Saraiva J, Barreiros E, Almeida A et al. Aspect-oriented software maintenance metrics: A systematic mapping study. In Proc. the 16th International Conference on Evaluation & Assessment in Software Engineering, May 2012, pp.253-262.
[9]
Kumar B. A survey of key factors affecting software maintainability. In Proc. the 2012 International Conference on Computing Sciences, Sept. 2012, pp.261-266.
[10]

Tiwari G, Sharma A. Maintainability techniques for software development approaches — A systematic survey. Int. J. Comput. Appl. Special Issue on Issues and Challenges in Networking, Intelligence and Computing Technologies, 2012, ICNICT(4): 28-31.

[11]

Saini M, Chauhan M. A roadmap of software system maintainability models. Int. J. Softw. Web Sci., 2013, 2(3): 69-73.

[12]
Vern R, Dubey S K. A review on appraisal techniques for web based maintainability. In Proc. the 5th Int. Conf. Conflu. Next Gener. Inf. Technol. Summit, Sept. 2014, pp.795-799.
[13]

Rajendra K, Namrata D. Maintainability quantification of object oriented design: A revisit. Int. J. Adv. Res. Comput. Sci. Softw. Eng., 2014, 4(12): 461-466.

[14]
Elmidaoui S, Cheikhi L, Idri A. Software product maintainability prediction: A survey of secondary studies. In Proc. the 4th International Conference on Control, Decision and Information Technologies, April 2017, pp.702-707.
[15]
Zhang D, Tsai J J P. Machine Learning Applications in Software Engineering. World Scientific Publishing Co., 2005.
[16]

Idri A, Amazal F A, Abran A. Analogy-based software development effort estimation: A systematic mapping and review. Inf. Softw. Technol., 2015, 58: 206-230.

[17]
Kitchenham B, Charters S. Guidelines for performing systematic literature reviews in software engineering. Technical Report, Keele University and University of Durham, 2007. http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=AE5CA418E8C8629591E397A8ECF1450E?doi=10.1.1.117.471&rep=rep1&type=pdf, Dec. 2019.
[18]

Wen J, Li S, Lin Z, Hu Y, Huang C. Systematic literature review of machine learning based software development effort estimation models. Inf. Softw. Technol., 2012, 54(1): 41-59.

[19]
Muthanna S, Kontogiannis K, Ponnambalam K, Stacey B. A maintainability model for industrial software systems using design level metrics. In Proc. the 7th Working Conference on Reverse Engineering, November 2000, pp.248-256.
[20]
Genero M, Piattini M, Manso E, Cantone G. Building UML class diagram maintainability prediction models based on early metrics. In Proc. the 9th International Symposium on Software Metrics, Sept. 2003, pp.263-275.
[21]
Sharma A, Grover P S, Kumar R. Predicting maintainability of component-based systems by using fuzzy logic. In Proc. the 2nd International Conference on Contemporary Computing, August 2009, pp.581-591.
[22]

Sharawat M S. Software maintainability prediction using neural networks. International Journal of Engineering Research and Applications, 2012, 2(2): 750-755.

[23]
Al-Jamimi H A, Ahmed M. Prediction of software maintainability using fuzzy logic. In Proc. the 3rd IEEE International Conference on Computer Science and Automation Engineering, June 2012, pp. 702-705.
[24]

Kaur A, Kaur K. Statistical comparison of modelling methods for software maintainability prediction. Int. J. Softw. Eng. Knowl. Eng., 2013, 23(6): 743-774.

[25]
Kaur A, Kaur K, Pathak K. Software maintainability prediction by data mining of software code metrics. In Proc. the 2014 International Conference on Data Mining and Intelligent Computing, Sept. 2014.
[26]
Wang L, Hu X, Ning Z, Ke W. Predicting object-oriented software maintainability using projection pursuit regression. In Proc. the 1st International Conference on Information Science and Engineering, Dec. 2009, pp.3827-3830.
[27]

Kaur A, Kaur K, Malhotra R. Soft computing approaches for prediction of software maintenance effort. Int. J. Comput. Appl., 2010, 1(16): 69-75.

[28]
Kaur A, Kaur K, Pathak K. A proposed new model for maintainability index of open source software. In Proc. the 3rd International Conference on Reliability, Infocom Technologies and Optimization, Oct. 2014.
[29]

Kumar L, Krishna A, Rath S K. The impact of feature selection on maintainability prediction of service-oriented applications. Serv. Oriented Comput. Appl., 2017, 11(2): 137-161.

[30]
Kumar L, Ashish S. A comparative study of different source code metrics and machine learning algorithms for predicting change proneness of object oriented systems. arXiv:1712.07944, 2017. https://arxiv.org/abs/1712.07944, Dec. 2019.
[31]

Kumar L, Rath S K. Hybrid functional link artificial neural network approach for predicting maintainability of objectoriented software. J. Syst. Softw., 2016, 121: 170-190.

[32]

Chug A, Malhotra R. Benchmarking framework for maintainability prediction of open source software using object oriented metrics. Int. J. Innov. Comput. Inf. Control, 2016, 12(2): 615-634.

[33]

Dubey S K, Rana A, Dash Y. Maintainability prediction of object-oriented software system by multilayer perceptron model. ACM SIGSOFT Softw. Eng. Notes, 2012, 37(5): 1-4.

[34]

Misra S C. Modeling design/coding factors that drive maintainability of software systems. Softw. Qual. J., 2005, 13(3): 297-320.

[35]
Jin X, Liu Y, Ren J, Xu A, Bie R. Locality preserving projection on source code metrics for improved software maintainability. In Proc. the 19th Australasian Joint Conference on Artificial Intelligence, Dec. 2006, pp.877-886.
[36]

Mittal H, Bhatia P. Software maintainability assessment based on fuzzy logic technique. ACM SIGSOFT Softw. Eng. Notes, 2009, 34(3): 1-5.

[37]
Dahiya S S, Chhabra J K, Kumar S. Use of genetic algorithm for software maintainability metrics’ conditioning. In Proc. the 15th International Conference on Advanced Computing and Communications, Dec. 2007, pp.87-92.
[38]
Pratap A, Chaudhary R, Yadav K. Estimation of software maintainability using fuzzy logic technique. In Proc. the 2014 International Conference on Issues and Challenges in Intelligent Computing Techniques, February 2014, pp.486-492.
[39]
Sandhya T, Anuradha C. Sequencing of refactoring techniques by Greedy algorithm for maximizing maintainability. In Proc. the 2016 International Conference on Advances in Computing, Communications and Informatics, September 2016, pp.1397-1403.
[40]

Szöke G, Antal G, Nagy C, Ferenc R, Gyimóthy T. Empirical study on refactoring large-scale industrial systems and its effects on maintainability. J. Syst. Softw., 2017, 129: 107-126.

[41]

Hegedüs P, Kádár I, Ferenc R, Gyimóthy T. Empirical evaluation of software maintainability based on a manually validated refactoring dataset. Inf. Softw. Technol., 2018, 95: 313-327.

[42]
Kiewkanya M, Jindasawat N, Muenchaisri P. A methodology for constructing maintainability model of objectoriented design. In Proc. the 4th International Conference on Quality Software, September 2004, pp.206-213.
[43]

Malhotra R. A systematic review of machine learning techniques for software fault prediction. Appl. Soft Comput., 2015, 27: 504-518.

[44]

Zhou Y, Leung H. Predicting object-oriented software maintainability using multivariate adaptive regression splines. J. Syst. Softw., 2007, 80(8): 1349-1361.

[45]

van Koten C, Gray A R. An application of Bayesian network for predicting object-oriented software maintainability. Inf. Softw. Technol., 2006, 48(1): 59-67.

[46]
Hayes J H, Zhao L. Maintainability prediction: A regression analysis of measures of evolving systems. In Proc. the 21st IEEE International Conference on Software Maintenance, Sept. 2005, pp.601-604.
[47]

Genero M, Manso E, Visaggio A, Canfora G, Piattini M. Building measure-based prediction models for UML class diagram maintainability. Empir. Softw. Eng., 2007, 12(5): 517-549.

[48]

Zhou Y, Xu B. Predicting the maintainability of open source software using design metrics. Wuhan Univ. J. Nat. Sci., 2008, 13(1): 14-20.

[49]
Rizvi S W A, Khan R A. Maintainability estimation model for object-oriented software in design phase (memood). arXiv:1004.4447, 2010. https://arxiv.org/pdf/1004.4447, Dec. 2019.
[50]
Tagoug N. Maintainability assessment in object-oriented system design. In Proc. the International Conference on Information Technology and e-Services, March 2012, pp.1-5.
[51]
Bakota T, Hegedüs P, Kortvelyesi P, Ferenc R, Gyimóthy T. A probabilistic software quality model. In Proc. the 27th IEEE International Conference on Software Maintenance, Sept. 2011, pp.243-252.
[52]

Al-Dallal J. Object-oriented class maintainability prediction using internal quality attributes. Inf. Softw. Technol., 2013, 55(11): 2028-2048.

[53]

Kumar R, Dhanda N. Maintainability measurement model for object-oriented design. International Journal of Advanced Research in Computer and Communication Engineering, 2015, 4(5): 68-71.

[54]
Malhotra R, Chug A. A metric suite for predicting software maintainability in data intensive applications. In Transactions on Engineering Technologies, Kim H K, Ao S L, Amouzegar M A (eds.), Springer, 2014, pp.161-175.
[55]
Misra S, Egoeze F. Framework for maintainability measurement of web application for efficient knowledge-sharing on Campus Intranet. In Proc. the 14th International Conference on Computational Science and Its Applications, June 2014, pp.649-662.
[56]

Elish M O, Aljamaan H, Ahmad I. Three empirical studies on predicting software maintainability using ensemble methods. Soft Comput., 2015, 19(9): 2511-2524.

[57]
Sandhya T, Anuradha C. Predicting maintainability of open source software using Gene Expression Programming and bad smells. In Proc. the 5th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions), Sept. 2016, pp.452-459.
[58]

Almugrin S, Albattah W, Melton A. Using indirect coupling metrics to predict package maintainability and testability. J. Syst. Softw., 2016, 121: 298-310.

[59]
Kumar L, Rath S K, Sureka A. Empirical analysis on effectiveness of source code metrics for predicting changeproneness. In Proc. the 10th Innovations in Software Engineering Conference, February 2017, pp.4-14.
[60]
Kanika G, Anuradha C. Evaluation of instance-based feature subset selection algorithm for maintainability prediction. In Proc. the International Conference on Advances in Computing, Communications and Informatics, Sept. 2017, pp.1482-1487.
[61]

Reddy B R, Ojha A. Performance of maintainability index prediction models: A feature selection based study. Evol. Syst., 2017, 10(2): 179-204.

[62]
Kumar L, Santanu K R, Sureka A. Using source code metrics and multivariate adaptive regression splines to predict maintainability of service oriented software. In Proc. the 18th IEEE International Symposium on High Assurance Systems Engineering, January 2017, pp.88-95.
[63]

Malhotra R, Jangra R. Prediction and assessment of change prone classes using statistical and machine learning techniques. J. Inf. Process. Syst., 2017, 13(4): 778-804.

[64]
Bakota T, Hegedüs P, Ladányi G, Körtvélyesi P, Ferenc R, Gyimóthy T. A cost model based on software maintainability. In Proc. the 28th IEEE International Conference on Software Maintenance, Sept. 2012, pp.316-325.
[65]

Hegedüs P, Bakota T, Ladányi G, Faragó C, Ferenc R. A drill-down approach for measuring maintainability at source code element level. Electronic Communication of the European Association of Software Science and Technology, 2013, 60: Article No. 2.

[66]
Shibata K, Rinsaka K, Dohi T, Okamura H. Quantifying software maintainability based on a faultdetection/correction model. In Proc. the 13th Pacific Rim International Symposium on Dependable Computing, Dec. 2007, pp.35-42.
[67]
di Lucca G A, Fasolino A R, Tramontana P, Visaggio C A. Towards the definition of a maintainability model for web applications. In Proc. the 8th European Conference on Software Maintenance and Reengineering, March 2004, pp.279-287.
[68]
ThwinMMT, Quah T S. Application of neural networks for estimating software maintainability using object-oriented metrics. In Proc. the 5th International Conference on Software Engineering and Knowledge Engineering, July 2003, pp.69-73.
[69]

Aggarwal K K, Singh Y, Kaur A, Malhotra R. Application of artificial neural network for predicting maintainability using object-oriented metrics. Int. J. Comput. Electr. Autom. Control Inf. Eng., 2008, 2(10): 3552-3556.

[70]
Tian Y, Chen C, Zhang C. AODE for source code metrics for improved software maintainability. In Proc. the 4th International Conference on Semantics, Knowledge and Grid, Dec. 2008, pp.330-335.
[71]

Olatunji S O, Rasheed Z, Sattar K A, Al-Mana A M, Alshayeb M, El-Sebakhy E A. Extreme learning machine as maintainability prediction model for object-oriented software systems. J. Comput., 2010, 2(8): 49-56.

[72]

Malhotra R, Chug A. Software maintainability prediction using machine learning algorithms. Softw. Eng. An Int. J., 2012, 2(2): 19-36.

[73]

Dash Y, Dubey S K, Rana A. Maintainability prediction of object oriented software system by using artificial neural network approach. Int. J. Soft Comput. Eng., 2012, 2(2): 420-423.

[74]
Aljamaan H, Elish M O, Ahmad I. An ensemble of computational intelligence models for software maintenance effort prediction. In Proc. the 12th International Work-Conference on Artificial Neural Networks, June 2013, pp.592-603.
[75]
Ye F, Zhu X, Wang Y. A new software maintainability evaluation model based on multiple classifiers combination. In Proc. the 2013 International Conference on Quality, Reliability, Risk, Maintenance, and Safety Engineering, July 2013, pp.1588-1591.
[76]

Ahmed M A, Al-Jamimi H A. Machine learning approaches for predicting software maintainability: A fuzzy-based transparent model. IET Softw., 2013, 7(6): 317-326.

[77]

Olatunji S O. Sensitivity-based linear learning method and extreme learning machines compared for software maintainability prediction of object-oriented software systems. ICTACT J. Soft Comput., 2013, 3(3): 514-523.

[78]

Malhotra R, Chug A. Application of group method of data handling model for software maintainability prediction using object oriented systems. Int. J. Syst. Assur. Eng. Manag., 2014, 5(2): 165-173.

[79]
Kumar L, Rath S K. Neuro—Genetic approach for predicting maintainability using Chidamber and Kemerer software metrics suite. In Proc. the 11th International Conference on Computing and Information Technology, July 2015, pp.31-40.
[80]

Kumar L, Naik D K, Rath S K. Validating the effectiveness of object-oriented metrics for predicting maintainability. Procedia Comput. Sci., 2015, 57: 798-806.

[81]
Jain A, Tarwani S, Chug A. An empirical investigation of evolutionary algorithm for software maintainability prediction. In Proc. the 2016 IEEE Students’ Conference on Electrical, Electronics and Computer Science, March 2016, pp.1-6.
[82]
Jin C, Liu J A. Applications of support vector mathine and unsupervised learning for predicting maintainability using object-oriented metrics. In Proc. the 2nd International Conference on Multimedia and Information Technology, April 2010, pp.24-27.
[83]

Chandra D. Support vector approach by using radial kernel function for prediction of software maintenance effort on the basis of multivariate approach. Int. J. Comput. Appl., 2012, 51(4): 21-25.

[84]

Kumar L, Kumar M, Rath S K. Maintainability prediction of web service using support vector machine with various kernel methods. Int. J. Syst. Assur. Eng. Manag., 2017, 8(2): 205-222.

[85]
Kumar L, Kumar S R, Sureka A. Using source code metrics to predict change-prone web services: A case-study on eBay services. In Proc. the 2017 IEEE Workshop on Machine Learning Techniques for Software Quality Evaluation, February 2017, pp.1-7.
[86]
Elish M O, Elish K O. Application of TreeNet in predicting object-oriented software maintainability: A comparative study. In Proc. the 13th European Conference on Software Maintenance and Reengineering, March 2009, pp.69-78.
[87]
Cai L, Liu Z, Zhang J, Tong W, Yang G. Evaluating software maintainability using fuzzy entropy theory. In Proc. the 9th IEEE/ACIS International Conference on Computer and Information Science, Aug. 2010, pp.737-742.
[88]

Dhankhar P, Mittal H, Mittal A. Maintainability prediction for object oriented software. Int. J. Adv. Eng. Sci., 2011, 1(1): 8-11.

[89]

Dubey S K, Rana A. A fuzzy approach for evaluation of maintainability of object oriented software system. Int. J. Comput. Appl., 2012, 49(21): 1-6.

[90]
Hao X L, Zhu X D, Liu L. Research on software maintainability evaluation based on fuzzy integral. In Proc. the 2013 International Conference on Quality, Reliability, Risk, Maintenance, and Safety Engineering, July 2013, pp.1279-1282.
[91]
Olatunji S O, Selamat A. Type-2 fuzzy logic based prediction model of object oriented software maintainability. In Proc. the 13th International Conference on Intelligent Software Methodologies, Tools and Techniques, Sept. 2015, pp.329-342.
[92]

Kundu S, Tyagi K. Maintainability assessment for software by using a hybrid fuzzy multi-criteria analysis approach. Manag. Sci. Lett., 2017, 7(6): 255-274.

[93]
Kumar L, Rath S K. Software maintainability prediction using hybrid neural network and fuzzy logic approach with parallel computing concept. Int. J. Syst. Assur. Eng. Manag., 2017, 8(S2): 1487-1502.
[94]

Yenduri G, Madhwaraj G. An authoritative method using fuzzy logic to evaluate maintainability index and utilizability of software. Adv. Model. Anal. B, 2017, 60(3): 566-580.

[95]
Yu H, Peng G, Liu W. An application of case based reasoning to predict structure maintainability. In Proc. the 2009 International Conference on Computational Intelligence and Software Engineering, Dec. 2009.
[96]

Mehra A, Dubey S K. Maintainability evaluation of objectoriented software system using clustering techniques. Int. J. Comput. Technol., 2013, 5(2): 136-143.

[97]

Lee C C, Chung P C, Tsai J R, Chang C I. Robust radial basis function neural networks. IEEE Trans. Syst. Man, Cybern. Part B, 1999, 29(6): 674-685.

[98]
Murphy K P. Naive Bayes classifiers. https://www.cs.ubc.ca/_murphyk/Teaching/CS340-Fall06/reading/NB.pdf, Dec. 2019.
[99]
Hegedüs P, Ladányi G, Siket I, Ferenc R. Towards building method level maintainability models based on expert evaluations. In Proc. the International Conferences on Computer Applications for Software Engineering, Disaster Recovery, and Business Continuity, Nov. 2012, pp.146-154.
[100]
Conte S D, Dunsmore H E, Shen Y E. Software Engineering Metrics and Models. Benjamin-Cummings Publishing Co., 1986.
[101]
Port D, Korte M. Comparative studies of the model evaluation criterions MMRE and PRED in software cost estimation research. In Proc. the 2nd ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, October 2008, pp.51-60.
[102]

MacDonell S G. Establishing relationships between specification size and software process effort in CASE environments. Inf. Softw. Technol., 1997, 39(1): 35-45.

[103]

Li W, Henry S. Object-oriented metrics that predict maintainability. J. Syst. Softw., 1993, 23(2): 111-122.

[104]
Elmidaoui S, Cheikhi L, Idri A. Accuracy comparison of empirical studies on software product maintainability prediction. In Proc. the World Conference on Information Systems and Technologies, March 2018, pp.26-35.
[105]

Shepperd M, Kadoda G. Comparing software prediction techniques using simulation. IEEE Trans. Softw. Eng., 2001, 27(11): 1014-1022.

[106]

Briand L C, Brasili V R, Hetmanski C J. Developing interpretable models with optimized set reduction for identifying high-risk software components. IEEE Trans. Softw. Eng., 1993, 19(11): 1028-1044.

[107]

Elmidaoui S, Cheikhi L, Idri A, Abran A. Empirical studies on software product maintainability prediction: A systematic mapping and review. e-Informatica Softw. Eng. J., 2019, 13(1): 141-202.

[108]
Keung J W. Theoretical maximum prediction accuracy for analogy-based software cost estimation. In Proc. the 15th Asia-Pacific Software Engineering Conference, Dec. 2008, pp.495-502.
Journal of Computer Science and Technology
Pages 1147-1174
Cite this article:
Elmidaoui S, Cheikhi L, Idri A, et al. Machine Learning Techniques for Software Maintainability Prediction: Accuracy Analysis. Journal of Computer Science and Technology, 2020, 35(5): 1147-1174. https://doi.org/10.1007/s11390-020-9668-1

429

Views

11

Crossref

N/A

Web of Science

13

Scopus

0

CSCD

Altmetrics

Received: 24 April 2019
Revised: 26 February 2020
Published: 30 September 2020
©Institute of Computing Technology, Chinese Academy of Sciences 2020
Return