The pursuit of optimal neural network architectures is foundational to the progression of Neural Architecture Search (NAS). However, the existing NAS methods suffer from the following problem using traditional search strategies, i.e., when facing a large and complex search space, it is difficult to mine more effective architectures within a reasonable time, resulting in inferior search results. This research introduces the Generative Pre-trained Transformer NAS (GPT-NAS), an innovative approach designed to overcome the limitations which are inherent in traditional NAS strategies. This approach improves search efficiency and obtains better architectures by integrating GPT model into the search process. Specifically, we design a reconstruction strategy that utilizes the trained GPT to reorganize the architectures obtained from the search. In addition, to equip the GPT model with the design capabilities of neural architecture, we propose the use of the GPT model for training on a neural architecture dataset. For each architecture, the structural information of its previous layers is utilized to predict the next layer of structure, iteratively traversing the entire architecture. In this way, the GPT model can efficiently learn the key features required for neural architectures. Extensive experimental validation shows that our GPT-NAS approach beats both manually constructed neural architectures and automatically generated architectures by NAS. In addition, we validate the superiority of introducing the GPT model in several ways, and find that the accuracy of the neural architecture on the image dataset obtained from the search after introducing the GPT model is improved by up to about 9%.

As a high-tech strategic emerging comprehensive industry, the nuclear industry is committed to the research, production, and processing of nuclear fuel, as well as the development and utilization of nuclear energy. Nowadays, the nuclear industry has made remarkable progress in the application fields of nuclear weapons, nuclear power, nuclear medical treatment, radiation processing, and so on. With the development of artificial intelligence and the proposal of "Industry 4.0", more and more artificial intelligence technologies are introduced into the nuclear industry chain to improve production efficiency, reduce operation cost, improve operation safety, and realize risk avoidance. Meanwhile, deep learning, as an important technology of artificial intelligence, has made amazing progress in theoretical and applied research in the nuclear industry, which vigorously promotes the development of informatization, digitization, and intelligence of the nuclear industry. In this paper, we first simply comb and analyze the intelligent demand scenarios in the whole industrial chain of the nuclear industry. Then, we discuss the data types involved in the nuclear industry chain. After that, we investigate the research status of deep learning in the application fields corresponding to different data types in the nuclear industry. Finally, we discuss the limitation and unique challenges of deep learning in the nuclear industry and the future direction of the intelligent nuclear industry.