AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home International Journal of Crowd Science Article
PDF (588.4 KB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research paper | Open Access

Definitive guidelines toward effective mobile devices crowdtesting methodology

Qamar Naith( ), Fabio Ciravegna
Department of Computer Science, University of Sheffield, Sheffield, UK
Show Author Information

Abstract

Purpose

This paper aims to gauge developers’ perspectives regarding the participation of the public and anonymous crowd testers worldwide, with a range of varied experiences. It also aims to gather their needs that could reduce their concerns of dealing with the public crowd testers and increase the opportunity of using the crowdtesting platforms.

Design/methodology/approach

An online exploratory survey was conducted to gather information from the participants, which included 50 mobile application developers from various countries with diverse experiences across Android and iOS mobile platforms.

Findings

The findings revealed that a significant proportion (90%) of developers is potentially willing to perform testing via the public crowd testers worldwide. This on condition that several fundamental features were available, which enable them to achieve more realistic tests without artificial environments on large numbers of devices. The results also demonstrated that a group of developers does not consider testing as a serious job that they have to pay for, which can affect the gig-economy and global market.

Originality/value

This paper provides new insights for future research in the study of how acceptable it is to work with public and anonymous crowd workers, with varying levels of experience, to perform tasks in different domains and not only in software testing. In addition, it will assist individual or small development teams who have limited resources or who do not have thousands of testers in their private testing community, to perform large-scale testing of their products.

Keywords

Gig-economyCompatibility testingLarge-scale crowdtestingMobile app testingPublic and anonymous crowd testers

References

 
Afzal, W. (2007), “Metrics in software test planning and test design processes”.
 
Almeida, M., Bilal, M., Finamore, A., Leontiadis, I., Grunenberger, Y., Varvello, M. and Blackburn, J. (2018), “Chimp: crowdsourcing human inputs for mobile phones”, in Proceedings of the 2018 World Wide Web Conference, International World Wide Web Conferences Steering Committee, pp. 45-54.https://doi.org/10.1145/3178876.3186035
Crossref
 

Alvertis, I., Koussouris, S., Papaspyros, D., Arvanitakis, E., Mouzakitis, S., Franken, S., Kolvenbach, S. and Prinz, W. (2016), “User involvement in software development processes”, Procedia Computer Science, Vol. 97, pp. 73-83.
Crossref Google Scholar
 
Bayley, I. Flood, D. Harrison, R. and Martin, C. (2012), “Mobitest: a cross-platform tool for testing mobile applications”.
 
Bell, E., Bryman, A. and Harley, B. (2018), Business Research Methods, Oxford university press.
 

Brabham, D.C. (2008), “Moving the crowd at istockphoto: the composition of the crowd and motivations for participation in a crowdsourcing application”, First Monday, Vol. 13 No. 6.
Google Scholar
 
Bryman, A. and Burgess, B. (2002), Analyzing Qualitative Data, Routledge.https://doi.org/10.4324/9780203413081
Crossref
 
Denscombe, M. (2014), The Good Research Guide: For Small-Scale Social Research Projects, McGraw-Hill Education.
 
Guaiani, F. and Muccini, H. (2015), “Crowd and laboratory testing, can they co-exist? An exploratory study”, 2015 IEEE/ACM 2nd International Workshop on CrowdSourcing in Software Engineering’, IEEE, pp. 32-37.https://doi.org/10.1109/CSI-SE.2015.14
Crossref
 

Holzer, A. and Ondrus, J. (2011), “Mobile application market: a developers perspective”, Telematics and Informatics, Vol. 28 No. 1, pp. 22-31.
Crossref Google Scholar
 
Huang, J-F. (2014), “Appacts: mobile app automated compatibility testing service”, In Mobile Cloud Computing, Services, and Engineering (MobileCloud), 2014 2nd IEEE International Conference on, IEEE, pp. 85-90.https://doi.org/10.1109/MobileCloud.2014.13
Crossref
 
Huang, J-F. and Gong, Y-Z. (2012), “Remote mobile test system: a mobile phone cloud for application testing”, in 2012 IEEE 4th International Conference on Cloud Computing Technology and Science (CloudCom), IEEE, pp. 1-4.https://doi.org/10.1109/CloudCom.2012.6512429
Crossref
 
Kaasila, J., Ferreira, D., Kostakos, V. and Ojala, T. (2012), “Testdroid: automated remote UI testing on android”, Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia, ACM, p. 28.https://doi.org/10.1145/2406367.2406402
Crossref
 
Kaufmann, N., Schulze, T. and Veit, D. (2011), “More than fun and money. Worker motivation in crowdsourcing – a study on mechanical Turk”,Amcis, Vol. 11, Detroit, pp. 1-11.
 
Knott, D. (2015), Hands-on Mobile App Testing, Pearson education Inc.
 

Marwell, G., Oliver, P.E. and Prahl, R. (1988), “Social networks and collective action: a theory of the critical mass. iii”, American Journal of Sociology, Vol. 94 No. 3, pp. 502-534.
Crossref Google Scholar
 
Mathers, N.J., Fox, N.J. and Hunn, A. (1998), Surveys and Questionnaires, NHS Executive, Trent.
 

Mazumdar, S., Wrigley, S. and Ciravegna, F. (2017), “Citizen science and crowdsourcing for earth observations: an analysis of stakeholder opinions on the present and future”,Remote Sensing, Vol. 9 No. 1, p. 87.
Crossref Google Scholar
 
Muntés-Mulero, V., Paladini, P., Manzoor, J., Gritti, A., Larriba-Pey, J.-L. and Mijnhardt, F. (2012), “Crowdsourcing for industrial problems”, International Workshop on Citizen in Sensor Networks, Springer, pp. 6-18.https://doi.org/10.1007/978-3-642-36074-9_2
Crossref
 

Naith, Q. and Ciravegna, F. (2018a), “Mobile devices compatibility testing strategy via crowdsourcing”, International Journal of Crowd Science, Vol. 2 No. 3, pp. 225-246.
Crossref Google Scholar
 
Naith, Q. and Ciravegna, F. (2018b), “Hybrid crowd-powered approach for compatibility testing of mobile devices and applications”, Proceedings of the 3rd International Conference on Crowd Science and Engineering, ACM, p. 1.https://doi.org/10.1145/3265689.3265690
Crossref
 
Naith, Q. and Ciravegna, F. (2019c), “The key considerations in building a crowd-testing platform for software developers”, Proceedings of the 4th International Conference on Crowd Science and Engineering, pp. 50-57.https://doi.org/10.1145/3371238.3371247
Crossref
 
Onwuzurike, L. and De Cristofaro, E. (2015), “Danger is my middle name: experimenting with SSL vulnerabilities in android apps”, Proceedings of the 8th ACM Conference on Security and Privacy in Wireless and Mobile Networks, ACM, p. 15.https://doi.org/10.1145/2766498.2766522
Crossref
 
Prathibhan, M., Malini, A., Venkatesh, N. and Sundarakantham, K. (2014), “An automated testing framework for testing android mobile applications in the cloud”, 2014 International Conference on Advanced Communication Control and Computing Technologies (ICACCCT), IEEE, pp. 1216-1219.https://doi.org/10.1109/ICACCCT.2014.7019292
Crossref
 
Robert, L. and Romero, D.M. (2015), “Crowd size, diversity and performance”, in Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, pp. 1379-1382.https://doi.org/10.1145/2702123.2702469
Crossref
 
Rosson, M.B., Carroll, J.M. and Hill, N. (2002), Usability Engineering: Scenario Based Development of Human-Computer Interaction, Morgan Kaufmann.
 
Sánchez-Charles, D., Nin, J., Solé, M. and Muntés-Mulero, V. (2014), “Worker ranking determination in crowdsourcing platforms using aggregation functions”, 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), IEEE, pp. 1801-1808.https://doi.org/10.1109/FUZZ-IEEE.2014.6891807
Crossref
 
Starov, O. (2013), “Cloud platform for research crowdsourcing in mobile testing”.
 
Vasilescu, B., Filkov, V. and Serebrenik, A. (2013), “Stackoverflow and GitHub: associations between software development and crowdsourced knowledge”, 2013 International Conference on Social Computing, IEEE, pp. 188-195.https://doi.org/10.1109/SocialCom.2013.35
Crossref
International Journal of Crowd Science
Volume 4 Issue 2,
June 2020
Pages 209-228
DOI: 10.1108/IJCS-01-2020-0002
Cite this article:
Naith Q, Ciravegna F. Definitive guidelines toward effective mobile devices crowdtesting methodology. International Journal of Crowd Science, 2020, 4(2): 209-228. https://doi.org/10.1108/IJCS-01-2020-0002

624

Views

14

Downloads

4

Crossref

4

Scopus

Altmetrics
Received: 21 January 2020
Revised: 25 February 2020
Accepted: 25 February 2020
Published: 28 April 2020
© The author(s)

Qamar Naith and Fabio Ciravegna. Published in International Journal of Crowd Science. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
Return