AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Computer Science and Technology Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Regular Paper

An Empirical Comparison Between Tutorials and Crowd Documentation of Application Programming Interface

School of Software, Dalian University of Technology, Dalian 116000, China
Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian 116000, China
School of Computer Science & Technology, Beijing Institute of Technology, Beijing 100000, China
Show Author Information

Abstract

API (application programming interface) documentation is critical for developers to learn APIs. However, it is unclear whether API documentation indeed improves the API learnability for developers. In this paper, we focus on two types of API documentation, i.e., official API tutorials and API crowd documentation. First, we analyze API coverage and check API consistencies in API documentation based on the API traceability. Then, we conduct a survey and extract several characteristics to analyze which API documentation can help developers learn APIs. Our findings show that: 1) API crowd documentation can be regarded as a supplement to the official API tutorials to some extent; 2) the concerns for frequently-used APIs between different types of API documentation show a huge mismatch, which may prevent developers from deeply understanding the usages of APIs through only one type of API documentation; 3) official API tutorials can help developers seek API information on a long page and API crowd documentation could provide long codes for a particular programming task. These findings may help developers select the suitable API documentation and find the useful information they need.

Keywords

quantitative analysisempirical studyAPI documentation

Electronic Supplementary Material

Download File(s)
jcst-36-4-856-Highlights.pdf (399.7 KB)

References

[1]
Subramanian S, Inozemtseva L, Holmes R. Live API documentation. In Proc. the 36th Int. Conf. Softw. Eng., May 2014, pp.643-652. DOI: 10.1145/2568225.2568313.
Crossref
[2]
Petrosyan G, Robillard M P, De Mori R. Discovering information explaining API types using text classification. In Proc. the 37th IEEE/ACM Int. Conf. Softw. Eng., May 2015, pp.869-879. DOI: 10.1109/ICSE.2015.97.
Crossref
[3]

Maalej W, Robillard M P. Patterns of knowledge in API reference documentation. IEEE Trans. Softw. Eng., 2013, 39(9): 1264-1282. DOI: 10.1109/TSE.2013.12.
Crossref Google Scholar
[4]

Robillard M P. What makes APIs hard to learn? Answers from developers. IEEE Softw., 2009, 26(6): 27-34. DOI: 10.1109/MS.2009.193.
Crossref Google Scholar
[5]
Thayer K. Using program analysis to improve API learnability. In Proc. the 2018 ACM Conf. Int. Computing Education Research, Aug. 2018, pp.292-293. DOI: 10.1145/3230977.3231009.
Crossref
[6]
Jiang J, Koskinen J, Ruokonen A, Systa T. Constructing usage scenarios for API redocumentation. In Proc. the 15th IEEE Int. Conf. Prog. Comprehension, Jun. 2007, pp.259-264. DOI: 10.1109/ICPC.2007.16.
Crossref
[7]
Jiang H, Zhang J, Li X, Ren Z, Lo D. A more accurate model for finding tutorial segments explaining APIs. In Proc. the 23rd IEEE Int. Conf. Softw. Analysis, Evolution, and Reengineering, Mar. 2016, pp.157-167. DOI: 10.1109/SANER.2016.59.
Crossref
[8]

Zhang J, Jiang H, Ren Z, Chen X. Recommending APIs for API related questions in Stack Overflow. IEEE Access, 2018, 6: 6205-6219. DOI: 10.1109/ACCESS.2017.2777845.
Crossref Google Scholar
[9]
Treude C, Storey M A. Effective communication of software development knowledge through community portals. In Proc. the 19th ACM SIGSOFT Symposium and the 13th European Conf. Foundations of Softw. Eng., Sept. 2011, pp.91-101. DOI: 10.1145/2025113.2025129.
Crossref
[10]

Robillard M P, Deline R. A field study of API learning obstacles. Empir. Softw. Eng., 2011, 16(6): 703-732. DOI: 10.1007/s10664-010-9150-8.
Crossref Google Scholar
[11]

Scaffidi C. Why are APIs difficult to learn and use. ACM Crossroads Student Magazine, 2006, 12(4): 4-10. DOI: 10.1145/1144359.1144363.
Crossref Google Scholar
[12]
Jiang H, Zhang J, Ren Z, Zhang T. An unsupervised approach for discovering relevant tutorial fragments for APIs. In Proc. the 39th IEEE/ACM Int. Conf. Softw. Eng., May 2017, pp.38-48. DOI: 10.1109/ICSE.2017.12.
Crossref
[13]
Ye X, Shen H, Ma X, Bunescu R, Liu C. From word embeddings to document similarities for improved information retrieval in software engineering. In Proc. the 38th Int. Conf. Softw. Eng., May 2016, pp.404-415. DOI: 10.1145/2884781.2884862.
Crossref
[14]

Uddin G, Robillard M P. How API documentation fails. IEEE Softw., 2015, 32(4): 68-75. DOI: 10.1109/MS.2014.80.
Crossref Google Scholar
[15]
Zhou Y, Gu R, Chen T, Huang Z, Panichella S, Gall H. Analyzing APIs documentation and code to detect directive defects. In Proc. the 39th IEEE/ACM Int. Conf. Softw. Eng., May 2017, pp.27-37. DOI: 10.1109/ICSE.2017.11.
Crossref
[16]
Parnin C, Treude C, Grammel L, Storey M A. Crowd documentation: Exploring the coverage and the dynamics of API discussions on Stack Overflow. Technical Report, Georgia Institute of Technology, 2012. http://chrisparnin.me/pdf/crowddoc.pdf, Jan. 2021.
[17]

Wang X, Huang C, Yao L, Benatallah B, Dong M. A survey on expert recommendation in community question answering. Journal of Computer Science and Tech., 2018, 33(4): 625-653. DOI: 10.1007/s11390-018-1845-0.
Crossref Google Scholar
[18]
Mamykina L, Manoim B, Mittal M, Hripcsak G, Hartmann B. Design lessons from the fastest Q&A site in the west. In Proc. the 2011 Annual Conf. Human Factors in Computing Systems, May 2011, pp.2857-2866. DOI: 10.1145/1978942.1979366.
Crossref
[19]
Beyer S, Macho C, Pinzger M. On Android API classes and their references on Stack Overflow. Technical Report, University of Klagenfurt, 2016. https://serg.aau.at/pub/StefanieBeyer/Publications/techreport_AAU-SERG-2016-0-01.pdf, Jan. 2021.
[20]

Zhang Y, Lo D, Xia X, Sun J L. Multi-factor duplicate question detection in Stack Overflow. Journal of Computer Science and Tech., 2015, 30(5): 981-997. DOI: 10.1007/s11390-015-1576-4.
Crossref Google Scholar
[21]
Yang X L, Lo D, Xia X, Wan Z Y, Sun J L. What security questions do developers ask? A large-scale study of Stack Overflow posts. Journal of Computer Science and Tech., 2016, 31(5): 910-924. DOI: 10.1007/s11390-016-1672-0.
Crossref
[22]

Rosen C, Shihab E. What are mobile developers asking about? A large scale study using Stack Overflow. Empir. Softw. Eng., 2016, 21(3): 1192-1223. DOI: 10.1007/s10664-015-9379-3.
Crossref Google Scholar
[23]

Barua A, Thomas S W, Hassan A E. What are developers talking about? An analysis of topics and trends in Stack Overflow. Empir. Softw. Eng., 2014, 19(3): 619-654. DOI: 10.1007/s10664-012-9231-y.
Crossref Google Scholar
[24]

Chen C, Wu K, Srinivasan V, Bharadwaj R K. The best answers? Think twice: Identifying commercial campaigns in the CQA forums. Journal of Computer Science and Tech., 2015, 30(4): 810-828. DOI: 10.1007/s11390-015-1562-x.
Crossref Google Scholar
[25]

Brito G, Hora A C, Valente M T, Romain R. On the use of replacement messages in API deprecation: An empirical study. Journal Syst. Softw., 2018, 137: 306-321. DOI: 10.1016/j.jss.2017.12.007.
Crossref Google Scholar
[26]
Dagenais B, Robillard M P. Recovering traceability links between an API and its learning resources. In Proc. the 34th Int. Conf. Softw. Eng., June 2012, pp.47-57. DOI: 10.1109/ICSE.2012.6227207.
Crossref
[27]
Rastkar S, Murphy G C, Murray G. Summarizing software artifacts: A case study of bug reports. In Proc. the 32nd ACM/IEEE Int. Conf. Softw. Eng., May 2010, pp.505-514. DOI: 10.1145/1806799.1806872.
Crossref
[28]
Bettenburg N, Just S, Schröter A, Weiss C, Premraj R, Zimmermann T. What makes a good bug report? In Proc. the 16th ACM SIGSOFT Int. Symposium on Foundations of Softw. Eng., November 2008, pp.308-318. DOI: 10.1145/1453101.1453146.
Crossref
[29]
Schneider T D. Information theory primer with an appendix on logarithms. http://users.fred.net/tds/lab/papers/primer/, Jan. 2021.
[30]
Smith E A, Senter R J. Automated readability index. Wright-Patterson Air Force Base, 1967. https://apps.dtic.mil/dtic/tr/fulltext/u2/667273.pdf, Jan. 2021.
[31]

Jay G T, Hale J E, Smith R K, Hale D P, Kraft N A, Ward C. Cyclomatic complexity and lines of code: Empirical evidence of a stable linear relationship. Journal of Softw. Eng. and Applications, 2009, 2(3): 137-143. DOI: 10.4236/jsea.2009.23020.
Crossref Google Scholar
[32]
Nykaza J, Messinger R, Boehme F, Norman CL, Mace M, Gordon M. What programmers really want: Results of a needs assessment for SDK documentation. In Proc. the 20th Annual ACM SIGDOC Int. Conf. Computer Documentation, Oct. 2002, pp.133-141. DOI: 10.1145/584955.584976.
Crossref
[33]

Mclellan S G, Roesler A W, Tempest J T, Spinuzzi C I. Building more usable APIs. IEEE Softw., 1998, 15(3): 78-86. DOI: 10.1109/52.676963.
Crossref Google Scholar
[34]

Santos A L, Myers B A. Design annotations to improve API discoverability. Journal of Systems and Softw., 2017, 126: 17-33. DOI: 10.1016/j.jss.2016.12.036.
Crossref Google Scholar
[35]

McCabe T J. A complexity measure. IEEE Trans. Softw. Eng., 1976, SE-2(4): 308-320. DOI: 10.1109/TSE.197-6.233837.
Crossref Google Scholar
[36]
Yuan T, Thung F, Sharma A, Lo D. APIBot: Question answering bot for API documentation. In Proc. the 32nd IEEE/ACM Int. Conf. Automated Softw. Eng., Oct. 30-Nov. 3, 2017, pp.153-158. DOI: 10.1109/ASE.2017.8115628.
Crossref
[37]
Zar J H. Spearman rank correlation. Encyclopedia of Bio-statistics. DOI: 10.1002/0470011815.b2a15150.
Crossref
[38]
Mandelin D, Xu L, Bodík R, Kimelman D. Jungloid mining: Helping to navigate the API jungle. In Proc. the ACM SIG-PLAN Conf. Prog. Language Design and Implementation, June 2005, 40(6): 48-61. DOI: 10.1145/1065010.1065018.
Crossref
[39]

Mann H B, Whitney D R. On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 1947, 18(1): 50-60. DOI: 10.1214/AOMS/1177730491.
Crossref Google Scholar
[40]
Kitchenham B A, Peeger S L. Personal opinion surveys. In Guide to Advanced Empirical Software Engineering, Shull F, Singer J, Sjøberg D (eds.), Springer, 2008, pp.63-92. DOI: 10.1007/978-1-84800-044-5_3.
Crossref
[41]

Zou W, Lo D, Chen Z, Xia X, Feng Y, Xu B. How practitioners perceive automated bug report management techniques. IEEE Trans. Softw. Eng., 2020, 46(8): 836-862. DOI: 10.1109/TSE.2018.2870414.
Crossref Google Scholar
[42]
David L O, Nagappan N, Zimmermann T. How practitioners perceive the relevance of software engineering research. In Proc. the 10th Joint Meeting on Foundations of Soft. Eng., Aug. 30-Sept. 4, 2015, pp.415-425. DOI: 10.1145/2786805.2786809.
Crossref
[43]

Fisher R A. On the interpretation of χ2 from contingency tables, and the calculation of P. Journal of the Royal Statistical Society, 1922, 85(1): 87-94. DOI: 10.1111/j.2397-2335.1922.tb00768.x.
Crossref Google Scholar
[44]
McDonald J H. Handbook of Biological Statistics (3rd edition). Sparky House Publisher, 2009.
[45]
Rocha A M, Maia M A. Automated API documentation with tutorials generated from Stack Overflow. In Proc. the 30th Brazilian Symposium on Softw. Eng., Sept. 2016, pp.33-42. DOI: 10.1145/2973839.2973847.
Crossref
[46]

Kim J, Lee S, Hwang S, Kim S. Enriching documents with examples: A corpus mining approach. ACM Trans. Information Systems, 2013, 31(1): Article No. 1. DOI: 10.1145/2414782.2414783.
Crossref Google Scholar
[47]
Treude C, Robillard M P. Augmenting API documentation with insights from Stack Overflow. In Proc. the 38th IEEE/ACM Int. Conf. Softw. Eng., May 2016, pp.392-403. DOI: 10.1145/2884781.2884800.
Crossref
Journal of Computer Science and Technology
Volume 36 Issue 4,
July 2021
Pages 856-876
DOI: 10.1007/s11390-020-0042-0
Cite this article:
Tang Y-X, Ren Z-L, Jiang H, et al. An Empirical Comparison Between Tutorials and Crowd Documentation of Application Programming Interface. Journal of Computer Science and Technology, 2021, 36(4): 856-876. https://doi.org/10.1007/s11390-020-0042-0

351

Views

0

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 18 September 2019
Accepted: 17 May 2020
Published: 05 July 2021
©Institute of Computing Technology, Chinese Academy of Sciences 2021
Return