AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (18.6 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Towards Data-Driving Multi-View Evaluation Framework for Scratch

Department of Computer Science, Beijing University of Posts and Telecommunication, Beijing 100876, China
Intellectual Property Institute, Shandong Polytechnic College, Jining 272067, China
Show Author Information

Abstract

As one of the most popular visual programming languages, Scratch has a lot of evaluation around it. Reasonable evaluation can help programmers understand their projects better. At the same time, it can also provide a reference for them to browse other projects in the online community. Most of the existing evaluations on Scratch are carried from three perspectives: Computational Thinking (CT) ability, visual presentation aesthetics, and code quality. Among them, the assessment of CT and code quality is mainly carried out from the program script, while the evaluation of visual aesthetics is analyzed from the perspective of image sequences generated by project execution. The single-view evaluation focuses on the performance of a program in a certain aspect and is one-sided. In this paper, we propose a multi-view evaluation framework to integrate various evaluations using different policies. We quantitatively analyze the assessment of different views driven by data. Combined with overall evaluations that represent human opinions, we analyze their differences and connections. Through experiments, we determine the weights of different integration policies, the proposed multi-view evaluation method can generate evaluation results similar to human opinions.

Keywords

machine learningScratchstatic analysisComputational Thinking (CT)visual aestheticintegration policy

References

[1]
K. Brennan and M. Resnick, New frameworks for studying and assessing the development of computational thinking, in Proc. 2012 Ann. Meeting of the American Educational Research Association, Vancouver, Canada, 2012, p. 25.
Google Scholar
[2]
L. Seiter and B. Foreman, Modeling the learning progressions of computational thinking of primary grade students, in Proc. Ninth Ann. Int. ACM Conf. Int. Computing Education Research, San Diego, CA, USA, 2013, pp. 59–66.
Crossref Google Scholar
[3]
J. M. Wing, Computational thinking and thinking about computing, Philosoph. Trans. Roy. Soc. A Math. Phys. Eng. Sci., vol. 366, no. 1881, pp. 3717–3725, 2008.
Crossref Google Scholar
[4]
X. Chai, Y. Sun, H. Luo, and M. Guizani, DWES: A dynamic weighted evaluation system for scratch based on computational thinking, IEEE Trans. Emerg. Top. Comput., vol. 10, no. 2, pp. 917–932, 2022.
Google Scholar
[5]
X. Lu, Z. Lin, H. Jin, J. Yang, and J. Z. Wang, RAPID: Rating pictorial aesthetics using deep learning, in Proc. 22nd ACM Int. Conf. Multimedia, Orlando, FL, USA, 2014, pp. 457–466.
Crossref Google Scholar
[6]
X. Lu, Z. Lin, X. Shen, R. Mech, and J. Z. Wang, Deep multi-patch aggregation network for image style, aesthetics, and quality estimation, in Proc. 2015 IEEE Int. Conf. Computer Vision, Santiago, Chile, 2015, pp. 990–998.
Crossref Google Scholar
[7]
S. Ma, J. Liu, and C. W. Chen, A-Lamp: Adaptive layout-aware multi-patch deep convolutional neural network for photo aesthetic assessment, in Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 722–731.
Crossref Google Scholar
[8]
K. Sheng, W. Dong, C. Ma, X. Mei, F. Huang, and B. G. Hu, Attention-based multi-patch aggregation for image aesthetic assessment, in Proc. 26th ACM Int. Conf. Multimedia, Seoul, Republic of Korea, 2018, pp. 879–886.
Crossref Google Scholar
[9]
Z. Wang, S. Chang, F. Dolcos, D. Beck, D. Liu, and T. S. Huang, Brain-inspired deep networks for image aesthetics assessment, arXiv preprint arXiv: 1601.04155, 2016.
Google Scholar
[10]
J. Moreno-León, G. Robles, and M. Román-González, Dr. Scratch: Automatic analysis of scratch projects to assess and foster computational thinking, RED. Revist. Edu. Distan, vol. 46, no. 10, pp. 1–23, 2015.
Google Scholar
[11]
Z. Chang, Y. Sun, T. Y. Wu, and M. Guizani, Scratch analysis tool (SAT): A modern scratch project analysis tool based on ANTLR to assess computational thinking skills, in Proc 14th Int. Wireless Communications & Mobile Computing Conf. (IWCMC), Limassol, Cyprus, 2018, pp. 950–955.
Crossref Google Scholar
[12]
S. Kong, X. Shen, Z. Lin, R. Mech, and C. Fowlkes, Photo aesthetics ranking network with attributes and content adaptation, in Proc. 14th European Conf. Computer Vision, Amsterdam, The Netherlands, 2016, pp. 662–679.
Crossref Google Scholar
[13]
K. Schwarz, P. Wieschollek, and H. P. A. Lensch, Will people like your image? Learning the aesthetic space, in Proc. 2018 IEEE Winter Conf. Applications of Computer Vision (WACV), Lake Tahoe, NV, USA, 2018, pp. 2048–2057.
Crossref Google Scholar
[14]
N. Murray, L. Marchesotti, and F. Perronnin, AVA: A large-scale database for aesthetic visual analysis, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Providence, RI, USA, 2012, pp. 2408–2415.
Crossref Google Scholar
[15]
X. Lu, Z. Lin, H. Jin, J. Yang, and J. Z. Wang, Rating image aesthetics using deep learning, IEEE Trans. Multimedia, vol. 17, no. 11, pp. 2021–2034, 2015.
Crossref Google Scholar
[16]
V. Hosu, B. Goldlucke, and D. Saupe, Effective aesthetics prediction with multi-level spatially pooled features, in Proc. 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019, pp. 9375–9383.
Crossref Google Scholar
[17]
H. Talebi and P. Milanfar, NIMA: Neural image assessment, IEEE Trans. Image Process., vol. 27, no. 8, pp. 3998–4011, 2018.
Crossref Google Scholar
[18]
X. Chai, Y. Sun, H. Luo, and M. Guizani, An artistic analysis model based on sequence cartoon images for scratch, Int.J. Intellig. Syst., vol. 37, no. 11, pp. 9598–9619, 2022.
Crossref Google Scholar
[19]
C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, Inception-v4, inception-ResNet and the impact of residual connections on learning, in Proc. Thirty-First AAAI Conf. Artificial Intelligence, San Francisco, CA, USA, 2017, pp. 4278–4284.
Crossref Google Scholar
[20]
P. Techapalokul and E. Tilevich, Quality hound—an online code smell analyzer for scratch programs, in Proc. 2017 IEEE Symp. Visual Languages and Human-Centric Computing (VL/HCC), Raleigh, NC, USA, 2017, pp. 337–338.
Crossref Google Scholar
[21]
G. Fraser, U. Heuer, N. Körber, F. Obermüller, and E. Wasmeier, LitterBox: A linter for scratch programs, in Proc. IEEE/ACM 43rd Int. Conf. Software Engineering: Software Engineering Education and Training (ICSE-SEET), Madrid, Spain, 2021, pp. 183–188.
Crossref Google Scholar
[22]
F. J. M. Jr, The Kolmogorov-Smirnov test for goodness of fit, J. Am. Stat. Assoc., vol. 46, no. 253, pp. 68–78, 1951.
Crossref Google Scholar
[23]
J. R. Quinlan, Induction of decision trees, Mach. Learn., vol. 1, no. 1, pp. 81–106, 1986.
Crossref Google Scholar
[24]
L. Breiman, Random forests, Mach. Learn., vol. 45, no. 1, pp. 5–32, 2001.
Crossref Google Scholar
[25]
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning representations by back-propagating errors, Nature, vol. 323, no. 6088, pp. 533–536, 1986.
Crossref Google Scholar
[26]
J. H. Friedman, Greedy function approximation: A gradient boosting machine, Ann. Stat., vol. 29, no. 5, pp. 1189–1232, 2001.
Crossref Google Scholar
[27]
T. Q. Chen and C. Guestrin, XGBoost: A scalable tree boosting system, in Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, San Francisco, CA, USA, 2016, pp. 785–794.
Crossref Google Scholar
[28]
G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T. Y. Liu, LightGBM: A highly efficient gradient boosting decision tree, in Proc. 31st Int. Conf. Neural Information Processing Systems, Long Beach, CA, USA, 2017, pp. 3149–3157.
Google Scholar
Tsinghua Science and Technology
Volume 29 Issue 2,
April 2024
Pages 517-528
DOI: 10.26599/TST.2023.9010016
Cite this article:
Chai X, Sun Y, Gao Y. Towards Data-Driving Multi-View Evaluation Framework for Scratch. Tsinghua Science and Technology, 2024, 29(2): 517-528. https://doi.org/10.26599/TST.2023.9010016

438

Views

27

Downloads

0

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 10 January 2023
Revised: 16 March 2023
Accepted: 17 March 2023
Published: 22 September 2023
© The author(s) 2024.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return