AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Nano Research Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article

Accurate atomic scanning transmission electron microscopy analysis enabled by deep learning

Tianshu Chu1,2,3,§, Lei Zhou1,2,3,§, Bowei Zhang1,2,3( ), Fu-Zhen Xuan1,2,3( )
Shanghai Key Laboratory of Intelligent Sensing and Detection Technology, East China University of Science and Technology, Shanghai 200237, China
School of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237, China
Key Laboratory of Pressure Systems and Safety of Ministry of Education, East China University of Science and Technology, Shanghai 200237, China

§ Tianshu Chu and Lei Zhou contributed equally to this work.

Show Author Information

Graphical Abstract

Metal single atoms and atomic vacancies greatly affect the intrinsic activity of materials, but they are difficult to accurately identify by scanning transmission electron microscopy (STEM). In this work, we build up a universal model to analyze the vacancy defects and single atoms accurately and rapidly in experimental STEM images using a full convolution network.

Abstract

Currently, the machine learning (ML)-based scanning transmission electron microscopy (STEM) analysis is limited in the simulative stage, its application in experimental STEM is needed but challenging. Herein, we built up a universal model to analyze the vacancy defects and single atoms accurately and rapidly in experimental STEM images using a full convolution network. In our model, the unavoidable interference factors of noise, aberration, and carbon contamination were fully considered during the training, which were difficult to be considered in the past. Even toward the simultaneous identification of various vacancy types and low-contrast single atoms in the low-quality STEM images, our model showed rapid process speed (45 images per second) and high accuracy (> 95%). This work represents an improvement in experimental STEM image analysis by ML.

Keywords

deep learninglow-dimensional materialsatomic defectssingle atoms

Electronic Supplementary Material

Download File(s)
12274_2023_6104_MOESM1_ESM.pdf (3.1 MB)

References

[1]

Chu, T. S.; Rong, C.; Zhou, L.; Mao, X. Y.; Zhang, B. W.; Xuan, F. Z. Progress and perspectives of single-atom catalysts for gas sensing. Adv. Mater. 2023, 35, 2206783.
Crossref Google Scholar
[2]

Mitterreiter, E.; Schuler, B.; Micevic, A.; Hernangómez-Pérez, D.; Barthelmi, K.; Cochrane, K. A.; Kiemle, J.; Sigger, F.; Klein, J.; Wong, E. et al. The role of chalcogen vacancies for atomic defect emission in MoS2. Nat. Commun. 2021, 12, 3822.
Crossref Google Scholar
[3]

Hong, J. H.; Hu, Z. X.; Probert, M.; Li, K.; Lv, D. H.; Yang, X. N.; Gu, L.; Mao, N. N.; Feng, Q. L.; Xie, L. M. et al. Exploring atomic defects in molybdenum disulphide monolayers. Nat. Commun. 2015, 6, 6293.
Crossref Google Scholar
[4]

Sun, H. M.; Yan, Z. H.; Liu, F. M.; Xu, W. C.; Cheng, F. Y.; Chen, J. Self-supported transition-metal-based electrocatalysts for hydrogen and oxygen evolution. Adv. Mater. 2020, 32, 1806326.
Crossref Google Scholar
[5]

Zhang, N. Q.; Ye, C. L.; Yan, H.; Li, L. C.; He, H.; Wang, D. S.; Li, Y. D. Single-atom site catalysts for environmental catalysis. Nano Res. 2020, 13, 3165–3182.
Crossref Google Scholar
[6]

Zhang, Q. Q.; Guan, J. Q. Applications of single-atom catalysts. Nano Res. 2022, 15, 38–70.
Crossref Google Scholar
[7]

Zhang, J. Q.; Zhao, Y. F.; Guo, X.; Chen, C.; Dong, C. L.; Liu, R. S.; Han, C. P.; Li, Y. D.; Gogotsi, Y.; Wang, G. X. Single platinum atoms immobilized on an MXene as an efficient catalyst for the hydrogen evolution reaction. Nat. Catal. 2018, 1, 985–992.
Crossref Google Scholar
[8]

Du, Q. H.; Wu, L. J.; Cao, H. B.; Kang, C. J.; Nelson, C.; Pascut, G. L.; Besara, T.; Siegrist, T.; Haule, K.; Kotliar, G. et al. Vacancy defect control of colossal thermopower in FeSb2. npj Quantum Mater. 2021, 6, 13.
Crossref Google Scholar
[9]

Gao, W.; Li, S.; He, H. C.; Li, X. N.; Cheng, Z. X.; Yang, Y.; Wang, J. L.; Shen, Q.; Wang, X. Y.; Xiong, Y. J. et al. Vacancy-defect modulated pathway of photoreduction of CO2 on single atomically thin AgInP2S6 sheets into olefiant gas. Nat. Commun. 2021, 12, 4747.
Crossref Google Scholar
[10]

Dou, Y. H.; He, C. T.; Zhang, L.; Yin, H. J.; Al-Mamun, M.; Ma, J. M.; Zhao, H. J. Approaching the activity limit of CoSe2 for oxygen evolution via Fe doping and Co vacancy. Nat. Commun. 2020, 11, 1664.
Crossref Google Scholar
[11]

Sun, Y. H.; Zhao, H. F.; Zhou, D.; Zhu, Y. C.; Ye, H. Y.; Moe, Y. A.; Wang, R. M. Direct observation of epitaxial alignment of Au on MoS2 at atomic resolution. Nano Res. 2019, 12, 947–954.
Crossref Google Scholar
[12]

Gu, H. Y.; Liu, X.; Liu, X. F.; Ling, C. C.; Wei, K.; Zhan, G. M.; Guo, Y. B.; Zhang, L. Z. Adjacent single-atom irons boosting molecular oxygen activation on MnO2. Nat. Commun. 2021, 12, 5422.
Crossref Google Scholar
[13]

Hobbs, C.; Downing, C.; Jaskaniec, S.; Nicolosi, V. TEM and EELS characterization of Ni-Fe layered double hydroxide decompositions caused by electron beam irradiation. npj 2D Mater. Appl. 2021, 5, 29.
Crossref Google Scholar
[14]

Zhai, P. L.; Xia, M. Y.; Wu, Y. Z.; Zhang, G. H.; Gao, J. F.; Zhang, B.; Cao, S. Y.; Zhang, Y. T.; Li, Z. W.; Fan, Z. Z. et al. Engineering single-atomic ruthenium catalytic sites on defective nickel-iron layered double hydroxide for overall water splitting. Nat. Commun. 2021, 12, 4587.
Crossref Google Scholar
[15]

Cui, W. J.; Hu, Z. Y.; Unocic, R. R.; Van Tendeloo, G.; Sang, X. H. Atomic defects, functional groups and properties in MXenes. Chin. Chem. Lett. 2021, 32, 339–344.
Crossref Google Scholar
[16]

Zhang, B. W.; Zhu, C. Q.; Wu, Z. S.; Stavitski, E.; Lui, Y. H.; Kim, T. H.; Liu, H.; Huang, L.; Luan, X. C.; Zhou, L. et al. Integrating Rh species with NiFe-layered double hydroxide for overall water splitting. Nano Lett. 2020, 20, 136–144.
Crossref Google Scholar
[17]

Pattengale, B.; Huang, Y. C.; Yan, X. X.; Yang, S. Z.; Younan, S.; Hu, W. H.; Li, Z. D.; Lee, S.; Pan, X. Q.; Gu, J. et al. Dynamic evolution and reversibility of single-atom Ni(II) active site in 1T-MoS2 electrocatalysts for hydrogen evolution. Nat. Commun. 2020, 11, 4114.
Crossref Google Scholar
[18]

Kalinin, S. V.; Ophus, C.; Voyles, P. M.; Erni, R.; Kepaptsoglou, D.; Grillo, V.; Lupini, A. R.; Oxley, M. P.; Schwenker, E.; Chan, M. K. Y. et al. Machine learning in scanning transmission electron microscopy. Nat. Rev. Methods Primers 2022, 2, 11.
Crossref Google Scholar
[19]
Jimenez-Del-Toro, O.; Otálora, S.; Andersson, M.; Eurén, K.; Hedlund, M.; Rousson, M.; Müller, H.; Atzori, M. Analysis of histopathology images: From traditional machine learning to deep learning. In Biomedical Texture Analysis. Depeursinge, A.; Al-Kadi, A. S.; Mitchell, J. R., Eds.; Elsevier: Amsterdam, 2017; pp 281–314.
[20]

Castiglioni, I.; Rundo, L.; Codari, M.; Di Leo, G.; Salvatore, C.; Interlenghi, M.; Gallivanone, F.; Cozzi, A.; D’Amico, N. C.; Sardanelli, F. AI applications to medical images: From machine learning to deep learning. Phys. Medica 2021, 83, 9–24.
Crossref Google Scholar
[21]

Lecun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 1998, 86, 2278–2324.
Crossref Google Scholar
[22]
Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. In 3rd International Conference on Learning Representations, San Diego, USA, 2015, pp 1–14.
[23]
Liu, J. J.; Hou, Q. B.; Cheng, M. M.; Feng, J. S.; Jiang, J. M. A simple pooling-based design for real-time salient object detection. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019, pp 3912–3921.
[24]

Vaisman, R.; Kroese, D. P.; Gertsbakh, I. B. Improved sampling plans for combinatorial invariants of coherent systems. IEEE Trans. Reliab. 2016, 65, 410–424.
Crossref Google Scholar
[25]
Wang, T. T.; Borji, A.; Zhang, L. H.; Zhang, P. P.; Lu, H. C. A stagewise refinement model for detecting salient objects in images. In 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017, pp 4039–4048.
[26]

Cheng, M. M.; Fan, D. P. Structure-measure: A new way to evaluate foreground maps. Int. J. Comput. Vis. 2021, 129, 2622–2638.
Crossref Google Scholar
[27]
Zhang, P. P.; Wang, D.; Lu, H. C.; Wang, H. Y.; Yin, B. C. Learning uncertain convolutional features for accurate saliency detection. In 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017, pp 212–221.
[28]
Chen, S. H.; Tan, X. L.; Wang, B.; Hu, X. L. Reverse attention for salient object detection. In 15th European Conference on Computer Vision, Munich, Germany, 2018, pp 236–252.
[29]

Zhang, Z. X.; Liu, Q. J.; Wang, Y. H. Road extraction by deep residual U-Net. IEEE Geosci. Remote Sens. Lett. 2018, 15, 749–753.
Crossref Google Scholar
[30]

Qin, X. B.; Zhang, Z. C.; Huang, C. Y.; Dehghan, M.; Zaiane, O. R.; Jagersand, M. U2-Net: Going deeper with nested U-structure for salient object detection. Pattern Recognit. 2020, 106, 107404.
Crossref Google Scholar
[31]

Lee, C. H.; Khan, A.; Luo, D.; Santos, T. P.; Shi, C. Q.; Janicek, B. E.; Kang, S. M.; Zhu, W. J.; Sobh, N. A.; Schleife, A. et al. Deep learning enabled strain mapping of single-atom defects in two-dimensional transition metal dichalcogenides with sub-picometer precision. Nano Lett. 2020, 20, 3369–3377.
Crossref Google Scholar
[32]

Yang, S. H.; Choi, W.; Cho, B. W.; Agyapong-Fordjour, F. O. T.; Park, S.; Yun, S. J.; Kim, H. J.; Han, Y. K.; Lee, Y. H.; Kim, K. K. et al. Deep learning-assisted quantification of atomic dopants and defects in 2D materials. Adv. Sci. 2021, 8, 2101099.
Crossref Google Scholar
[33]

Lin, R. Q.; Zhang, R.; Wang, C. Y.; Yang, X. Q.; Xin, H. L. TEMImageNet training library and AtomSegNet deep-learning models for high-precision atom segmentation, localization, denoising, and deblurring of atomic-resolution images. Sci. Rep. 2021, 11, 5386.
Crossref Google Scholar
[34]

Aguiar, J. A.; Gong, M. L.; Unocic, R. R.; Tasdizen, T.; Miller, B. D. Decoding crystallography from high-resolution electron imaging and diffraction datasets with deep learning. Sci. Adv. 2019, 5, eaaw1949.
Crossref Google Scholar
[35]

Sang, X. H.; Xie, Y.; Lin, M. W.; Alhabeb, M.; Van Aken, K. L.; Gogotsi, Y.; Kent, P. R. C.; Xiao, K.; Unocic, R. R. Atomic defects in monolayer titanium carbide (Ti3C2Tx) MXene. ACS Nano 2016, 10, 9193–9200.
Crossref Google Scholar
[36]

Trentino, A.; Madsen, J.; Mittelberger, A.; Mangler, C.; Susi, T.; Mustonen, K.; Kotakoski, J. Atomic-level structural engineering of graphene on a mesoscopic scale. Nano Lett. 2021, 21, 5179–5185.
Crossref Google Scholar
[37]

Ziatdinov, M.; Dyck, O.; Maksov, A.; Li, X. F.; Sang, X. H.; Xiao, K.; Unocic, R. R.; Vasudevan, R.; Jesse, S.; Kalinin, S. V. Deep learning of atomically resolved scanning transmission electron microscopy images: Chemical identification and tracking local transformations. ACS Nano 2017, 11, 12742–12752.
Crossref Google Scholar
[38]

Zhao, L.; Pan, Y. L.; Wang, S.; Zhang, L.; Islam, M. S. A hybrid crack detection approach for scanning electron microscope image using deep learning method. Scanning 2021, 2021, 5558668.
Crossref Google Scholar
[39]

Mitchell, S.; Parés, F.; Faust Akl, D.; Collins, S. M.; Kepaptsoglou, D. M.; Ramasse, Q. M.; Garcia-Gasulla, D.; Pérez-Ramírez, J.; López, N. Automated image analysis for single-atom detection in catalytic materials by transmission electron microscopy. J. Am. Chem. Soc. 2022, 144, 8018–8029.
Crossref Google Scholar
[40]

Qi, K.; Cui, X. Q.; Gu, L.; Yu, S. S.; Fan, X. F.; Luo, M. C.; Xu, S.; Li, N. B.; Zheng, L. R.; Zhang, Q. H. et al. Single-atom cobalt array bound to distorted 1T MoS2 with ensemble effect for hydrogen evolution catalysis. Nat. Commun. 2019, 10, 5231.
Crossref Google Scholar
[41]

Addou, R.; Colombo, L.; Wallace, R. M. Surface defects on natural MoS2. ACS Appl. Mater. Interfaces 2015, 7, 11921–11929.
Crossref Google Scholar
[42]

Rasool, H. I.; Ophus, C.; Zettl, A. Atomic defects in two dimensional materials. Adv. Mater. 2015, 27, 5771–5777.
Crossref Google Scholar
[43]

Stobinski, L.; Lesiak, B.; Malolepszy, A.; Mazurkiewicz, M.; Mierzwa, B.; Zemek, J.; Jiricek, P.; Bieloshapka, I. Graphene oxide and reduced graphene oxide studied by the XRD, TEM and electron spectroscopy methods. J. Electron Spectros. Relat. Phenomena 2014, 195, 145–154.
Crossref Google Scholar
[44]

Peng, W.; Han, J. H.; Lu, Y. R.; Luo, M.; Chan, T. S.; Peng, M.; Tan, Y. W. A general strategy for engineering single-metal sites on 3D porous N, P Co-doped Ti3C2Tx MXene. ACS Nano 2022, 16, 4116–4125.
Crossref Google Scholar
[45]

Zhao, D.; Chen, Z.; Yang, W. J.; Liu, S. J.; Zhang, X.; Yu, Y.; Cheong, W. C.; Zheng, L. R.; Ren, F. Q.; Ying, G. B. et al. MXene (Ti3C2) vacancy-confined single-atom catalyst for efficient functionalization of CO2. J. Am. Chem. Soc. 2019, 141, 4086–4093.
Crossref Google Scholar
[46]

Cheng, Y. W.; Dai, J. H.; Zhang, Y. M.; Song, Y. Transition metal modification and carbon vacancy promoted Cr2CO2 (MXenes): A new opportunity for a highly active catalyst for the hydrogen evolution reaction. J. Mater. Chem. A 2018, 6, 20956–20965.
Crossref Google Scholar
[47]

Wan, Q.; Li, S. N.; Liu, J. B. First-principle study of Li-ion storage of functionalized Ti2C monolayer with vacancies. ACS Appl. Mater. Interfaces 2018, 10, 6369–6377.
Crossref Google Scholar
[48]

Wu, H.; Guo, Z. L.; Zhou, J.; Sun, Z. M. Vacancy-mediated lithium adsorption and diffusion on MXene. Appl. Surf. Sci. 2019, 488, 578–585.
Crossref Google Scholar
[49]

Hao, J. C.; Zhuang, Z. C.; Cao, K. C.; Gao, G. H.; Wang, C.; Lai, F. L.; Lu, S. L.; Ma, P. M.; Dong, W. F.; Liu, T. X. et al. Unraveling the electronegativity-dominated intermediate adsorption on high-entropy alloy electrocatalysts. Nat. Commun. 2022, 13, 2662.
Crossref Google Scholar
[50]

Chu, C. H.; Huang, D. H.; Gupta, S.; Weon, S.; Niu, J. F.; Stavitski, E.; Muhich, C.; Kim, J. H. Neighboring Pd single atoms surpass isolated single atoms for selective hydrodehalogenation catalysis. Nat. Commun. 2021, 12, 5179.
Crossref Google Scholar
[51]

Zhang, C. Y.; Liang, S. X.; Liu, W.; Eickemeyer, F. T.; Cai, X. B.; Zhou, K.; Bian, J. M.; Zhu, H. W.; Zhu, C.; Wang, N. et al. Ti1-graphene single-atom material for improved energy level alignment in perovskite solar cells. Nat. Energy 2021, 6, 1154–1163.
Crossref Google Scholar
[52]

Li, Z.; Xiao, Y.; Chowdhury, P. R.; Wu, Z. W.; Ma, T.; Chen, J. Z.; Wan, G.; Kim, T. H.; Jing, D. P.; He, P. L. et al. Direct methane activation by atomically thin platinum nanolayers on two-dimensional metal carbides. Nat. Catal. 2021, 4, 882–891.
Crossref Google Scholar
[53]

Bao, H. H.; Qiu, Y.; Peng, X. Y.; Wang, J. A.; Mi, Y. Y.; Zhao, S. Z.; Liu, X. J.; Liu, Y. F.; Cao, R.; Zhuo, L. C. et al. Isolated copper single sites for high-performance electroreduction of carbon monoxide to multicarbon products. Nat. Commun. 2021, 12, 238.
Crossref Google Scholar
[54]

Madsen, J.; Liu, P.; Kling, J.; Wagner, J. B.; Hansen, T. W.; Winther, O.; Schiøtz, J. A deep learning approach to identify local structures in atomic-resolution transmission electron microscopy images. Adv. Theory Simul. 2018, 1, 1800037.
Crossref Google Scholar
[55]

Seo, S. Y.; Yang, D. H.; Moon, G.; Okello, O. F. N.; Park, M. Y.; Lee, S. H.; Choi, S. Y.; Jo, M. H. Identification of point defects in atomically thin transition-metal dichalcogenide semiconductors as active dopants. Nano Lett. 2021, 21, 3341–3354.
Crossref Google Scholar
[56]

Maksov, A.; Dyck, O.; Wang, K.; Xiao, K.; Geohegan, D. B.; Sumpter, B. G.; Vasudevan, R. K.; Jesse, S.; Kalinin, S. V.; Ziatdinov, M. Deep learning analysis of defect and phase evolution during electron beam-induced transformations in WS2. npj Comput. Mater. 2019, 5, 12.
Crossref Google Scholar
[57]
PyTorch. https://pytorch.org/docs/stable/generated/torch.nn.BCELoss.
[58]
Kirkland, E. J. Advanced Computing in Electron Microscopy; 2nd ed. Springer: New York, 2010; pp 1–351.
Nano Research
Volume 17 Issue 4,
April 2024
Pages 2971-2980
DOI: 10.1007/s12274-023-6104-1
Cite this article:
Chu T, Zhou L, Zhang B, et al. Accurate atomic scanning transmission electron microscopy analysis enabled by deep learning. Nano Research, 2024, 17(4): 2971-2980. https://doi.org/10.1007/s12274-023-6104-1
Topics:
Catalyst activityChemical sensors High resolution transmission electron microscopyImage processing

1071

Views

4

Crossref

4

Web of Science

4

Scopus

0

CSCD

Altmetrics
Received: 11 July 2023
Revised: 14 August 2023
Accepted: 16 August 2023
Published: 23 September 2023
© Tsinghua University Press 2023
Return