AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home International Journal of Crowd Science Article
PDF (994.5 KB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research paper | Open Access

Towards data-driven software engineering skills assessment

Jun Lin1( ), Han Yu2, Zhengxiang Pan3, Zhiqi Shen2, Lizhen Cui4
The Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly LILY, Nanyang Technological University, Singapore, and Alibaba-NTU Singapore Joint Research Institute, Singapore
School of Computer Science and Engineering, Nanyang Technological University NTU, Singapore
Interdisciplinary Graduate School, Nanyang Technological University NTU, Singapore
School of Computer Science and Technology, Shandong University, China
Show Author Information

Abstract

Purpose

Today’s software engineers often work in teams to develop complex software systems. Therefore, successful software engineering in practice require team members to possess not only sound programming skills such as analysis, design, coding and testing but also soft skills such as communication, collaboration and self-management. However, existing examination-based assessments are often inadequate for quantifying students’ soft skill development. The purpose of this paper is to explore alternative ways for assessing software engineering students’ skills through a data-driven approach.

Design/methodology/approach

In this paper, the exploratory data analysis approach is adopted. Leveraging the proposed online agile project management tool – Human-centred Agile Software Engineering (HASE), a study was conducted involving 21 Scrum teams consisting of over 100 undergraduate software engineering students in multi-week coursework projects in 2014.

Findings

During this study, students performed close to 170,000 software engineering activities logged by HASE. By analysing the collected activity trajectory data set, the authors demonstrate the potential for this new research direction to enable software engineering educators to have a quantifiable way of understanding their students’ skill development, and take a proactive approach in helping them improve their programming and soft skills.

Originality/value

To the best of the authors’ knowledge, there has yet to be published previous studies using software engineering activity data to assess software engineers’ skills.

Keywords

Crowd-sourced design and engineeringTask-oriented crowdsourcingAgile software engineeringTools and platforms to support crowd science and engineering

References

 
Anzai, Y. (1992), “Pattern recognition and machine learning”, Academic Press, Cambridge MA.https://doi.org/10.1016/B978-0-08-051363-8.50005-3
Crossref
 

Cockburn, A. and Highsmith, J. (2001), “Agile software development, the people factor”, Computer, Vol. 34 No. 11, pp. 131-133.
Crossref Google Scholar
 

Doan, A., Ramakrishnan, R. and Halevy, A.Y. (2011), “Crowdsourcing systems on the world-wide web”, Communications of the ACM, Vol. 54 No. 4, pp. 86-96.
Crossref Google Scholar
 
Heymann, P. and Garcia-Molina, H. (2011), “Turkalytics: analytics for human computation”, in Proceedings of the 20th International Conference on World Wide Web (WWW’11), pp. 477-486.https://doi.org/10.1145/1963405.1963473
Crossref
 

Jøsang, A., Ismail, R. and Boyd, C. (2007), “A survey of trust and reputation systems for online service provision”, Decision Support Systems (DSS), Vol. 43 No. 2, pp. 618-644.
Crossref Google Scholar
 
Kitchenham, B.A., Dyba, T. and Jorgensen, M. (2004), “Evidence-based software engineering”, in Proceedings of the 26th International Conference on Software Engineering (ICSE’04), pp. 273-281.
 

Likert, R. (1932), “A technique for the measurement of attitudes”, Archives of Psychology, Vol. 22 No. 140.
Google Scholar
 
Lin, J. (2013), “Context-aware task allocation for distributed agile team”, in Proceedings of the 28th IEEE/ACM International Conference on Automated Software Engineering (ASE’13), pp. 758-761.https://doi.org/10.1109/ASE.2013.6693151
Crossref
 
Liu, S., Yu, H., Miao, C. and Kot, A.C. (2013), “A fuzzy logic based reputation model against unfair ratings”, inProceedings of the 12th International Conference on Autonomous Agents and Multi-agent Systems (AAMAS’13), pp. 821-828.
 
Lin, J., Yu, H., Shen, Z. and Miao, C. (2014), “Studying task allocation decisions of novice agile teams with data from agile project management tools”, in Proceedings of the 29th IEEE/ACM International Conference on Automated Software Engineering (ASE’14), pp. 689-694.https://doi.org/10.1145/2642937.2642959
Crossref
 
Pan, L., Meng, X., Shen, Z. and Yu, H. (2009), “A reputation pattern for service oriented computing”, in Proceedings of the 7th International Conference on Information, Communications and Signal Processing (ICICS’09).
 

Salleh, N., Mendes, E. and Grundy, J. (2011), “Empirical studies of pair programming for CS/SE teaching in higher education: a systematic literature review”, IEEE Transactions on Software Engineering (TSE), Vol. 37 No. 4, pp. 509-525.
Crossref Google Scholar
 
Tukey, J.W. (1977), Exploratory Data Analysis, Addison-Wesley, Boston.
 
Wang, Y. and Singh, M.P. (2007), “Formal ‘trust’, model for multiagent systems”, in Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI’07), pp. 1551-1556.
 

Yu, H., Shen, Z., Leung, C., Miao, C. and Lesser, V.R. (2013b), “A survey of multi-agent trust management systems”, IEEE Access, Vol. 1 No. 1, pp. 35-50.
Crossref Google Scholar
 
Yu, H., Liu, S., Kot, A.C., Miao, C. and Leung, C. (2011), “Dynamic witness selection for trustworthy distributed cooperative sensing in cognitive radio networks”, in Proceedings of the 13th IEEE International Conference on Communication Technology (ICCT’11), pp. 1-6.https://doi.org/10.1109/ICCT.2011.6157821
Crossref
 
Yu, H., Miao, C., An, B., Leung, C. and Lesser, V.R. (2013a), “A reputation management approach for resource constrained trustee agents”, in Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI’13), pp. 418-424.
 
Yu, H., Miao, C., Shen, Z., Leung, C., Chen, Y. and Yang, Q. (2015), “Efficient task Sub-delegation for crowdsourcing”, in Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI-15), pp. 1305-1311.
 
Yu, H., Shen, Z., Miao, C. and An, B. (2013c), “A reputation-aware decision making approach for improving the efficiency of crowdsourcing systems”, in Proceedings of the 12th International Conference on Autonomous Agents and Multi-agent Systems (AAMAS’13), pp. 1315-1316.
 

Yu, H., Shen, Z., Miao, C., Leung, C. and Niyato, D. (2010), “A survey of trust and reputation management systems in wireless communications”, Proceedings of the IEEE, Vol. 98 No. 10, pp. 1755-1772.
Crossref Google Scholar
International Journal of Crowd Science
Volume 2 Issue 2,
November 2018
Pages 123-135
DOI: 10.1108/IJCS-07-2018-0014
Cite this article:
Lin J, Yu H, Pan Z, et al. Towards data-driven software engineering skills assessment. International Journal of Crowd Science, 2018, 2(2): 123-135. https://doi.org/10.1108/IJCS-07-2018-0014

680

Views

17

Downloads

3

Crossref

2

Scopus

Altmetrics
Received: 04 July 2018
Revised: 06 September 2018
Accepted: 07 September 2018
Published: 11 October 2018
© The author(s)

Jun Lin, Han Yu, Zhengxiang Pan, Zhiqi Shen and Lizhen Cui. Published in International Journal of Crowd Science. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
Return