AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Social Computing Article
PDF (1.8 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Leveraging Human-AI Collaboration in Crowd-Powered Source Search: A Preliminary Study

College of Systems Engineering, National University of Defense Technology, Changsha 430000, China
Show Author Information

Abstract

Source search is an important problem in our society, relating to finding fire sources, gas sources, or signal sources. Particularly, in an unexplored and potentially dangerous environment, an autonomous source search algorithm that employs robotic searchers is usually applied to address the problem. Such environments could be completely unknown and highly complex. Therefore, novel search algorithms have been designed, combining heuristic methods and intelligent optimization, to tackle search problems in large and complex search spaces. However, these intelligent search algorithms were not designed to address completeness and optimality, and therefore commonly suffer from the problems such as local optimums or endless loops. Recent studies have used crowd-powered systems to address the complex problems that cannot be solved by machines on their own. While leveraging human intelligence in an AI system has been shown to be effective in making the system more reliable, whether using the power of the crowd can improve autonomous source search algorithms remains unanswered. To this end, we propose a crowd-powered source search approach enabling human-AI collaboration, which uses human intelligence as external supports to improve existing search algorithms and meanwhile reduces human efforts using AI predictions. Furthermore, we designed a crowd-powered prototype system and carried out an experiment with both experts and non-experts, to complete 200 source search scenarios (704 crowdsourcing tasks). Quantitative and qualitative analysis showed that the sourcing search algorithm enhanced by crowd could achieve both high effectiveness and efficiency. Our work provides valuable insights in human-AI collaborative system design.

Keywords

crowdsourcingsource searchcrowd-powered systemscrowd computing

References

[1]
Y. Zhao, Z. Zhu, B. Chen, S. Qiu, Crowd-powered source searching in complex environments, presented at Chinese CSCW 2022: Computer Supported Cooperative Work and Social Computing, Taiyuan, China, 2022.
Crossref
[2]
B. Ristic, A. Skvortsov, and A. Gunatilaka, A study of cognitive strategies for an autonomous search, Inf. Fusion, vol. 28, pp. 1–9, 2016.
Crossref
[3]
M. Hutchinson, H. Oh, and W. -H. Chen, Entrotaxis as a strategy for autonomous search and source reconstruction in turbulent conditions, Inf. Fusion, vol. 42, pp. 179–189, 2018.
Crossref
[4]
S. Desale, A. Rasool, S. Andhale, and P. Rane, Heuristic and meta-heuristic algorithms and their relevance to the real world: A survey, Int. J. Comput. Eng. Res. Trends, vol. 2, no. 5, pp. 296–304, 2015.
[5]
S. Amershi, D. Weld, M. Vorvoreanu, A. Fourney, B. Nushi, P. Collisson, J. Suh, S. Iqbal, P. N. Bennett, K. Inkpen, et al., Guidelines for human-AI interaction, in Proc. 2019 CHI Conf. Human Factors in Computing Systems, Glasgow, UK, 2019, pp. 1–13.
Crossref
[6]
M. J. Franklin, D. Kossmann, T. Kraska, S. Ramesh, and R. Xin, CrowdDB: Answering queries with crowdsourcing, in Proc. 2011 ACM SIGMOD Int. Conf. Management of Data, Athens, Greece, 2011, pp. 61–72.
Crossref
[7]
A. Bozzon, M. Brambilla, and S. Ceri, Answering search queries with CrowdSearcher, in Proc. 21st Int. Conf. World Wide Web, Lyon, France, 2012, pp. 1009–1018.
Crossref
[8]

L. Bradeško, M. Witbrock, J. Starc, Z. Herga, M. Grobelnik, and D. Mladenić, Curious cat: Mobile, context-aware conversational crowdsourcing knowledge acquisition, ACM Trans. Inf. Syst., vol. 35, no. 4, pp. 1–46, 2017.
Crossref Google Scholar
[9]
T. -H. K. Huang, J. C. Chang, and J. P. Bigham, Evorus: A crowd-powered conversational assistant built to automate itself over time, in Proc. 2018 CHI Conf. Human Factors in Computing Systems, Montreal, Canada, 2018, p. 295.
Crossref
[10]

I. Arous, L. Dolamic, J. Yang, A. Bhardwaj, G. Cuccu, and P. Cudré-Mauroux, MARTA: Leveraging human rationales for explainable text classification, Proc. AAAI Conf. Artif. Intell., vol. 35, no. 7, pp. 5868–5876, 2021.
Crossref Google Scholar
[11]
Y. Zhang, Q. V. Liao, and R. K. E. Bellamy, Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making, in Proc. 2020 Conf. Fairness, Accountability, and Transparency, Barcelona, Spain, 2020, pp. 295–305.
Crossref
[12]
Q. V. Liao, D. Gruen, and S. Miller, Questioning the AI: Informing design practices for explainable AI user experiences, in Proc. 2020 CHI Conf. Human Factors in Computing Systems, Honolulu, HI, USA, 2020, pp. 1–15.
Crossref
[13]

L. V. Ahn and L. Dabbish, Designing games with a purpose, Commun. ACM, vol. 51, no. 8, pp. 58–67, 2008.
Crossref Google Scholar
[14]
J. Yang, T. Drake, A. Damianou, and Y. Maarek, Leveraging crowdsourcing data for deep active learning an application: Learning intents in alexa, in Proc. 2018 World Wide Web Conference, Lyon, France, 2018, pp. 23–32.
Crossref
[15]
S. Qiu, A. Psyllidis, A. Bozzon, and G. -J. Houben, Crowd-mapping urban objects from street-level imagery, in Proc. WWW '19: The World Wide Web Conference, San Francisco, CA, USA, 2019, pp. 1521–1531.
Crossref
[16]
S. S. Noorian, S. Qiu, A. Psyllidis, A. Bozzon, and G. -J. Houben, Detecting, classifying, and mapping retail storefronts using street-level imagery, in Proc. 2020 Int. Conf. Multimedia Retrieval, Dublin, Ireland, 2020, pp. 495–501.
[17]
B. Green and Y. Chen, Disparate interactions: An algorithm-in-the-loop analysis of fairness in risk assessments, in Proc. Conf. Fairness, Accountability, and Transparency, Atlanta, GA, USA, 2019, pp. 90–99.
Crossref
[18]

B. Green and Y. Chen, The principles and limits of algorithm-in-the-loop decision making, Proc. ACM Hum.-Comput. Interact., vol. 3, no. CSCW, pp. 1–24, 2019.
Crossref Google Scholar
[19]
M. A. Goodrich and A. C. Schultz, Human–Robot Interaction: A Survey, Foundations and Trends® in Human–Computer Interaction, vol. 1, no. 3, pp 203–275, 2008.
Crossref
[20]

A. Kolling, P. Walker, N. Chakraborty, K. Sycara, and M. Lewis, Human interaction with robot swarms: A survey, IEEE Trans. Hum. Mach. Syst., vol. 46, no. 1, pp. 9–26, 2015.
Crossref Google Scholar
[21]
S. Bashyal and G. K. Venayagamoorthy, Human swarm interaction for radiation source search and localization, in Proc. 2008 IEEE Swarm Intelligence Symp., St. Louis, MO, USA, 2008, pp. 1–8.
Crossref
[22]
D. J. Bruemmer, D. D. Dudenhoeffer, M. D. McKay, and M. O. Anderson, A robotic swarm for spill finding and perimeter formation, Tech. Rep., Idaho National Engineering and Environmental Lab, Idaho Falls, ID, USA, 2002.
[23]

J. Casper and R. R. Murphy, Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center, IEEE Trans. Syst. Man Cybern. B Cybern., vol. 33, no. 3, pp. 367–385, 2003.
Crossref Google Scholar
[24]

X. -X. Chen and J. Huang, Odor source localization algorithms on mobile robots: A review and future outlook, Robotics Auton. Syst., vol. 112, pp. 123–136, 2019.
Crossref Google Scholar
[25]

T. Jing, Q. -H. Meng, and H. Ishida, Recent progress and trend of robot odor source localization, IEEJ Trans. Electr. Electron. Eng., vol. 16, no. 7, pp. 938–953, 2021.
Crossref Google Scholar
[26]

Y. Zhao, B. Chen, X. Wang, Z. Zhu, Y. Wang, G. Cheng, R. Wang, R. Wang, M. He, and Y. Liu, A deep reinforcement learning based searching method for source localization, Inf. Sci., vol. 588, pp. 67–81, 2022.
Crossref Google Scholar
[27]

N. E. Humphries, N. Queiroz, J. R. M. Dyer, N. G. Pade, M. K. Musyl, K. M. Schaefer, D. W. Fuller, J. M. Brunnschweiler, T. K. Doyle, J. D. R. Houghton, et al., Environmental context explains Lévy and Brownian movement patterns of marine predators, Nature, vol. 465, no. 7301, pp. 1066–1069, 2010.
Crossref Google Scholar
[28]
L. Champagne, R. G. Carl, and R. Hill, Search theory, agent-based simulation, and u-boats in the Bay of Biscay, in Proc. 2003 Winter Simulation Conference, New Orleans, LA, USA, 2003, pp. 991–998.
[29]

A. R. Unwin, Search theory and applications, J. Oper. Res. Soc., vol. 33, no. 5, pp. 494–495, 1982.
Crossref Google Scholar
[30]

Y. Zhao, B. Chen, Z. Zhu, F. Chen, Y. Wang, and D. Ma, Entrotaxis-Jump as a hybrid search algorithm for seeking an unknown emission source in a large-scale area with road network constraint, Expert Syst. Appl., vol. 157, p. 113484, 2020.
Crossref Google Scholar
[31]

S. Pang and J. A. Farrell, Chemical plume source localization, IEEE Trans. Syst. Man Cybern. B Cybern., vol. 36, no. 5, pp. 1068–1080, 2006.
Crossref Google Scholar
[32]

M. Hutchinson, H. Oh, and W. -H. Chen, A review of source term estimation methods for atmospheric dispersion events using static or mobile sensors, Inf. Fusion, vol. 36, pp. 130–148, 2017.
Crossref Google Scholar
[33]
A. Marjovi, J. G. Nunes, L. Marques, and A. D. Almeida, Multi-robot exploration and fire searching, in Proc. 2009 IEEE/RSJ Int. Conf. Intelligent Robots and Systems, St. Louis, MO, USA, 2009, pp. 1929–1934.
Crossref
[34]

M. Dadgar, S. Jafari, and A. Hamzeh, A PSO-based multi-robot cooperation method for target searching in unknown environments, Neurocomputing, vol. 177, pp. 62–74, 2016.
Crossref Google Scholar
[35]

H. Tang, W. Sun, H. Yu, A. Lin, and M. Xue, A multirobot target searching method based on bat algorithm in unknown environments, Expert Syst. Appl., vol. 141, p. 112945, 2020.
Crossref Google Scholar
[36]

C. Andrieu, N. D. Freitas, A. Doucet, and M. I. Jordan, An introduction to MCMC for machine learning, Mach. Learn., vol. 50, no. 1, pp. 5–43, 2003.
Crossref Google Scholar
[37]
A. Doucet, N. Freitas, and N. Gordon, Sequential Monte Carlo Methods in Practice. New York, NY, USA: Springer, 2013.
Crossref
[38]

E. Schubert, J. Sander, M. Ester, H. P. Kriegel, and X. Xu, DBSCAN revisited, revisited: Why and how you should (still) use DBSCAN, ACM Trans. Database Syst., vol. 42, no. 3, pp. 1–21, 2017.
Crossref Google Scholar
[39]

M. Vergassola, E. Villermaux, and B. I. Shraiman, ‘Infotaxis’ as a strategy for searching without gradients, Nature, vol. 445, no. 7126, pp. 406–409, 2007.
Crossref Google Scholar
[40]
J. Brooke, SUS: A ‘quick and dirty’ usability scale, in Usability Evaluation in Industry, P. W. Jordan, B. Thomas, I. L. McClelland, and B. Weerdmeester, eds. London, UK: CRC Press, 1996, pp. 189–194.
[41]
J. R. Lewis and J. Sauro, The factor structure of the system usability scale, in Proc. First International Conference on Human Centered Design, San Diego, CA, USA, 2009, pp. 94–103.
Crossref
[42]

A. Bangor, P. T. Kortum, and J. T. Miller, An empirical evaluation of the system usability scale, Int. J. Hum. Comput. Interact., vol. 24, no. 6, pp. 574–594, 2008.
Crossref Google Scholar
[43]

T. A. Wright and R. Cropanzano, The happy/productive worker thesis revisited, Res. Pers. Hum. Resour. Manag., vol. 26, pp. 269–307, 2007.
Crossref Google Scholar
[44]

P. M. A. Desmet, M. H. Vastenburg, and N. Romero, Mood measurement with pick-a-mood: Review of current methods and design of a pictorial self-report scale, J Des. Res., vol. 14, no. 3, pp. 241–279, 2016.
Crossref Google Scholar
[45]

C. Beedie, P. Terry, and A. Lane, Distinctions between emotion and mood, Cogn. Emot., vol. 19, no. 6, pp. 847–878, 2005.
Crossref Google Scholar
[46]

P. Verduyn, I. V. Mechelen, and F. Tuerlinckx, The relation between event processing and the duration of emotional experience, Emotion, vol. 11, no. 1, pp. 20–28, 2011.
Crossref Google Scholar
[47]
M. Zhuang and U. Gadiraju, In what mood are you today?: An analysis of crowd workers’ mood, performance and engagement, in Proc. 10th ACM Conf. Web Science, Boston, MA, USA, 2019, pp. 373–382.
Crossref
[48]
R. J. Moore, R. Arar, G. -J. Ren, and M. H. Szymanski, Conversational UX design, in Proc. 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, Denver, CO, USA, 2017, pp. 492–497.
Crossref
[49]
S. Kim, J. Lee, and G. Gweon, Comparing data from chatbot and web surveys: Effects of platform and conversational style on survey response quality, in Proc. 2019 CHI Conf. Human Factors in Computing Systems, Glasgow, UK, 2019, pp. 1–12.
Crossref
[50]
S. Qiu, U. Gadiraju, and A. Bozzon, Improving worker engagement through conversational microtask crowdsourcing, in Proc. 2020 CHI Conf. Human Factors in Computing Systems, Honolulu, HI, USA, 2020, pp. 1–12.
Crossref
[51]
U. Gadiraju and J. Yang, What can crowd computing do for the next generation of AI systems? in Proc. Crowd Science Workshop: Remoteness, Fairness, and Mechanisms as Challenges of Data Supply by Humans for Automation co-located with 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada, 2020, pp. 7–13.
Journal of Social Computing
Volume 4 Issue 2,
June 2023
Pages 95-111
DOI: 10.23919/JSC.2023.0002
Cite this article:
Zhao Y, Zhu Z, Chen B, et al. Leveraging Human-AI Collaboration in Crowd-Powered Source Search: A Preliminary Study. Journal of Social Computing, 2023, 4(2): 95-111. https://doi.org/10.23919/JSC.2023.0002

591

Views

67

Downloads

2

Crossref

1

Scopus

Altmetrics
Received: 19 April 2023
Accepted: 31 May 2023
Published: 30 June 2023
© The author(s) 2023.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return