Home International Journal of Crowd Science Article
PDF (176.5 KB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Tables (6)
Table. I
Table. II
Table. III
Table. IV
Table. V
Show 1 more tables Hide 1 tables
Research paper | Open Access

Utilizing short version big five traits on crowdsouring

Kousaku Igawa1(), Kunihiko Higa2, Tsutomu Takamiya2
Innovation Management, Tokyo Institute of Technology, Tokyo, Japan
Tokyo Institute of Technology, Tokyo, Japan
Show Author Information

Abstract

Purpose

The purpose of this paper is to examine the efficacy of the Japanese ten-item personality inventory (TIPI-J), a short version of the big five (BF) questionnaire, on crowdsourcing. The BF traits are indicators of personality and are said to be an effective predictor of study performance in various occupations. BF can be used in crowdsourcing to predict crowd workers’ performance; however, it will be difficult to use in practice for two reasons like the time-and-effort issue and the bias issue. In this study, an empirical analysis is conducted on crowdsourcing to examine if TIPI-J can solve those issues.

Design/methodology/approach

To investigate the issues, two tasks are posted on a crowdsourcing provider. Both TIPI-J and full version BF are conducted before and after selecting crowd workers. Structural validity and convergence validity are tested with correlation analysis between before (TIPI-J) and after (full version BF) data to examine the bias issue. Additionally, those correlations are compared with previous study and significances are examined.

Findings

The correlations in “conscientiousness” is 0.45-0.50, respectively, compared with a previous study, those two correlations did not show significance. This indicates that no clear bias exists.

Originality/value

This is the first research to investigate the efficacy of TIPI-J on crowdsourcing and showed that TIPI-J can be a useful tool for predicting crowd workers’ performance and thus it can help to select appropriate crowd workers.

Keywords

Human resourceQuality evaluationWork performanceCrowdsourcingBig fiveTask assignment

References

 
Anderson, G., (1992), and Viswesvaran, C. “An update of the validity of personality scales in personnel selection: a meta-analysis of studies published after 1992”, 13th Annual Conference of the Society of Industrial and Organizational Psychology, Dallas.
 
Assemi, B. and Schlagwein, D. (2012), “Profile information and business outcomes of providers in electronic service marketplaces: an empirical investigation”, Australasian Conference on Information Systems (ACIS), ACIS, pp. 1-10.
 

Barrick, M.R. and Mount, M.K. (1991), “The big five personality dimensions and job performance: a meta‐analysis”, Personnel Psychology, Vol. 44 No. 1, pp. 1-26.

Crossref Google Scholar
 

Barrick, M.R., Mount, M.K. and Judge, T.A. (2001), “Personality and performance at the beginning of the new millennium: What do we know and where do we go next?”, International Journal of Selection and Assessment, Vol. 9 Nos 1/2, pp. 9-30.

Crossref Google Scholar
 
Brier, E. and Pearson, R. (2020), “Upwork’s SVP of marketing explain what it takes to perfect an offering that relies on people”, available at: https://techdayhq.com/community/articles/upwork-s-svp-of-marketing-explains-what-it-takes-to-perfect-an-offering-that-relies-on-people, (accessed 23 December 2018).
 
Costa, P.T. and McCrea, R.R. (1992), “Revised neo personality inventory (neo-pi-r) and neo-five-factor inventory (NEO-FFI)”, Psychological Assessment Resources.
 

Digman, J.M. and Shmelyov, A.G. (1996), “The structure of temperament and personality in Russian children”, Journal of Personality and Social Psychology, Vol. 71 No. 2, pp. 341-351.

Crossref Google Scholar
 
Downs, J.S., Holbrook, M.B., Sheng, S. and Cranor, L.F. (2010), “Are your participants gaming the system?: screening mechanical Turk workers”, Proceedings of the SIGCHI conference on human factors in computing systems, ACM, 2399-2402.https://doi.org/10.1145/1753326.1753688
Crossref
 

Faullant, R., Holzmann, P. and Schwarz, E.J. (2016), “everybody is invited but not everybody will come – the influence of personality dispositions on users’entry decisions for crowdsourcing competitions”, International Journal of Innovation Management, Vol. 20 No. 6, p. 1650044.

Crossref Google Scholar
 

Feist, G.J. (1998), “A Meta-analysis of personality in scientific and artistic creativity”, Personality and Social Psychology Review, Vol. 2 No. 4, pp. 290-309.

Crossref Google Scholar
 

Fiske, D.W. (1949), “Consistency of the factorial structures of personality ratings from different sources”, The Journal of Abnormal and Social Psychology, Vol. 44 No. 3, pp. 329-344.

Crossref Google Scholar
 
Fujishima, Y. Yamada, N. and Tsuji, H. (2005), “Construction of short form of five-factor personality questionnaire”.https://doi.org/10.2132/personality.13.231
Crossref
 

Goldberg, L.R. (1990), “An alternative” description of personality”: the big-five factor structure”, Journal of Personality and Social Psychology, Vol. 59 No. 6, pp. 1216-1229.

Crossref Google Scholar
 

Goldberg, L.R. (1999), “A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models”, Personality Psychology in Europe, Vol. 7 No. 1, pp. 7-28.

Google Scholar
 
Gong, Y. (2015), “Enabling flexible IT services by crowdsourcing: a method for estimating crowdsourcing participants”, Open and Big Data Management and Innovation, Springer, pp. 275-286.https://doi.org/10.1007/978-3-319-25013-7_22
Crossref
 

Gosling, S.D., Gaddis, S. and Vazire, S. (2007), “Personality impressions based on Facebook profiles”, Icwsm, Vol. 7, pp. 1-4.

Google Scholar
 

Gosling, S.D., Rentfrow, P.J. and Swann, W.B. Jr, (2003), “A very brief measure of the Big-Five personality domains”, Journal of Research in Personality, Vol. 37 No. 6, pp. 504-528.

Crossref Google Scholar
 

Howe, J. (2006), “The rise of crowdsourcing”, Wired Magazine, Vol. 14 No. 6, pp. 1-4.

Google Scholar
 

Huang, J.-H. and Yang, Y.-C. (2010), “The relationship between personality traits and online shopping motivations”, Social Behavior and Personality: An International Journal, Vol. 38 No. 5, pp. 673-679.

Crossref Google Scholar
 
Igawa, K., Higa, K. and Takamiya, T. (2016), “An exploratory study on estimating the ability of high skilled crowd workers”, 2016 5th IIAI International Congress on Advanced Applied Informatics (IIAI-AAI), 10-14 July 2016, pp. 735-740.https://doi.org/10.1109/IIAI-AAI.2016.13
Crossref
 

John, O.P. and Srivastava, S. (1999), “The big five trait taxonomy: history, measurement, and theoretical perspectives”, Handbook of Personality: Theory and Research, Vol. 2 No. 1999, pp. 102-138.

Google Scholar
 
Kazai, G., Kamps, J. and Milic-Frayling, N. (2011), “Worker types and personality traits in crowdsourcing relevance labels”, Proceedings of the 20th ACM international conference on Information and knowledge management, ACM, pp. 1941-1944.https://doi.org/10.1145/2063576.2063860
Crossref
 
Kittur, A., Chi, E.H. and Suh, B. (2008), “Crowdsourcing user studies with Mechanical Turk”, 2008: ACM, pp. 453-456.https://doi.org/10.1145/1357054.1357127
Crossref
 
Mourelatos, E. and Tzagarakis, M. (2016), “Worker’s cognitive abilities and personality traits as predictors of effective task performance in crowdsourcing tasks”,Proceedings of 5th ISCA/DEGA Workshop on Perceptual Quality of Systems (PQS 2016), pp. 112-116.https://doi.org/10.21437/PQS.2016-24
Crossref
 

Murakami, Y. (2003), “Big five and psychometric conditions for their extraction in Japanese”, The Japanese Journal of Personality, Vol. 11 No. 2, pp. 70-85.

Crossref Google Scholar
 
Murakami, Y. and Murakami, C. (2001), Big Five Handbook, Gakugei Tosho Co., Ltd.
 

Oshio, A., Abe, S., Cutrone, P. and Gosling, S.D. (2014), “Further validity of the Japanese version of the ten-item personality inventory (TIPI-J)”, Journal of Individual Differences, Vol. 35 No. 4.

Crossref Google Scholar
 

Rammstedt, B. and John, O.P. (2007), “Measuring personality in one minute or less: a 10-item short version of the big five inventory in English and German”, Journal of Research in Personality, Vol. 41 No. 1, pp. 203-212.

Crossref Google Scholar
 
Ross, J., Irani, L., Silberman, M., Zaldivar, A. and Tomlinson, B. (2010), “Who are the crowdworkers?: shifting demographics in mechanical Turk”, CHI’10 extended abstracts on Human factors in computing systems, ACM, pp. 2863-2872.https://doi.org/10.1145/1753846.1753873
Crossref
 

Schmidt, F.L. and Hunter, J.E. (1998), “The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings”, Psychological Bulletin, Vol. 124 No. 2, pp. 262-274.

Crossref Google Scholar
 
Shimonaka, Y., Nakazato, K., Gondo, Y. and Takayama, M. (1999), Revised NEO-Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) Manual for the Japanese Version, (In Japanese), Tokyo Shinri, Tokyo.
 
Snagajob (2017), “Snagajob appoints former Upwork CEO to board of directors”, available at: www.prnewswire.com/news-releases/snagajob-appoints-former-upwork-ceo-to-board-of-directors-300417689.html (accessed 31 December 2018).
 

Uchida, T. (2002), “Effects of the speech rate on speakers’ personality-trait impressions”, Japanese Journal of Psychology.

Google Scholar
 

Wada, S. (1996), “Construction of the big five scales of personality trait terms and concurrent validity with NPI”, The Japanese Journal of Psychology, Vol. 67 No. 1, pp. 61-67.

Crossref Google Scholar
 
Wiggins, J.S. (1996), The Five-Factor Model of Personality: Theoretical Perspectives, Guilford Press.
International Journal of Crowd Science
Volume 4 Issue 2,
June 2020
Pages 117-132
DOI: 10.1108/IJCS-11-2019-0031
Cite this article:
Igawa K, Higa K, Takamiya T. Utilizing short version big five traits on crowdsouring. International Journal of Crowd Science, 2020, 4(2): 117-132. https://doi.org/10.1108/IJCS-11-2019-0031
Metrics & Citations  
Article History
Copyright
Rights and Permissions
Return

Demographic data of crowd workers

Task 1 (n = 36) Task 2 (n = 68)
Characteristics Class Frequency (%) Frequency (%)
Age 0-19 0 0.0 1 1.5
20-29 8 22.2 12 17.6
30-39 13 36.1 18 26.5
40-49 9 25.0 26 38.2
50-59 5 13.9 10 14.7
60-69 1 2.8 1 1.5
Sex Male 9 25.0 20 29.4
Female 27 75.0 48 70.6
Occupation Part-time 2 5.6 10 14.7
Student 2 5.6 2 2.9
Company employee 9 25.0 14 20.6
Self-employed 9 25.0 19 27.9
Homemaker 6 16.7 9 13.2
Other 8 22.2 14 20.6

Summary of pre and post questionnaire data

Big five trait Task 1 (n = 36) Task 2 (n = 67)
Pre (TIPI-J) Post (full version) Pre (TIPI-J) Post (full version)
Mean SD Range Mean SD Range Mean SD Range Mean SD Range
Extraversion 9.39 2.91 5-14 53.00 9.88 34-69 8.1 3.1 2-14 47.7 9.40 34-71
Agreeableness 11.72 1.28 9-14 47.47 8.99 32-67 10.4 2.0 4-14 44.9 9.04 21-67
Conscietiousness 10.75 2.21 4-14 58.14 7.14 45-70 8.9 2.5 3-14 53.5 10.72 27-70
Neuroticism 9.58 2.62 4-14 49.67 10.61 31-66 8.9 2.6 2-14 48.3 9.52 31-66
Openness 10.42 2.58 2-14 56.03 7.68 38-75 9.2 2.8 2-14 54.8 8.90 32-75

Summary of pre and post big five trait

Big five trait Task 1 (n = 36) Task 2 (n = 67)
Notes: *p < 0.5; ** p < 0.01
Extraversion 0.77** 0.71**
Agreeableness 0.31 0.44**
Conscientiousness 0.45** 0.50**
Neuroticism 0.56** 0.59**
Openness 0.58** 0.50**

Correlation and significance Task 1, Task 2 and previous study

Big five trait Oshio et al. Study(n = 216) Task 1 (n = 36) Task 2 (n = 67)
cor cor cor
Notes: *p < 0.5; ** p < 0.01
Extraversion 0.84 0.77 0.71*
Agreeableness 0.47 0.31 0.44
Conscientiousness 0.64 0.45 0.50
Neuroticism 0.67 0.56 0.59
Openness 0.50 0.58 0.50

The average scores of post “conscientiousness” for the 10 and 20 per cent with the highest and lowest pre score.

Task 1 Task 2
(%) n The average of post “conscientiousness” score n The average of post “conscientiousness” score
Highest group Lowest group p Highest group Lowest group p
Notes: *p < 0.5; ** p < 0.01
10 4 62.8 48.3 0.009** 7 61.6 44.4 0.009**
20 7 59.6 52.3 0.041* 14 59.4 44.4 0.001**

Multiple regression analysis among post big score and CSP information

Task 1 (n = 36) Task 2 (n = 67)
Dependent variable Post (big five)“conscientiousness” score Post (big five)“conscientiousness” score
Notes: *p < 0.5; ** p < 0.01
Adjusted R2 0.525 0.389
F 4.424 6.757
Independent variable Co-eff Std. err p Co eff Std. err p
Pre (TIPI-J) score 1.809 0.485 0.001** 2.013 0.483 0.000**
The number of project done 0.007 0.086 0.431 0.011 0.046 0.820
The number of tasks completed 0.006 0.060 0.926 0.000 0.000 0.435
The average score of the ratings awarded by 0.568 3.979 0.888 −8.363 4.391 0.062
The number of skills claimed −0.452 0.305 0.154 0.573 0.326 0.085
const 36.554 18.192 0.058 74.091 22.488 0.002