AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Article Link
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article

DeepRadiation: An intelligent augmented reality platform for predicting urban energy performance just through 360 panoramic streetscape images utilizing various deep learning models

Ali Nakhaee( )Arefe Paydar
Department of Architecture, Architecture School, College of Fine Arts, University of Tehran, Tehran 1415564583, Iran
Show Author Information

Graphical Abstract

Abstract

Urban energy simulation is critical for understanding and managing energy performance in cities. In this research, we design a novel framework called DeepRadiation, to enable automatic urban environmental performance prediction. By incorporating deep learning strategies, DeepRadiation predicts solar radiation on an urban scale using just panoramic streetscape images without any 3D modeling and simulation. New York City was chosen as the case study for this research. DeepRadiation is comprised of three different deep learning models organized into two stages. The first stage, named DeepRadiation modeling, serves as the framework's brain. At this stage, solar radiation analysis was performed using a Pix2Pix model, a type of conditional generative adversarial networks (GANs). After extracting GIS data and performing energy simulation analysis to prepare the dataset, the Pix2Pix model was trained on 10000 paired panoramic depth images of streetscapes with only building blocks and related panoramic images of streetscapes with only solar radiation analysis. Two GAN generator evaluation measures named qualitative evaluation and quantitative evaluation were used to validate the trained Pix2Pix model. Both demonstrated high levels of accuracy (qualitative evaluation: 93%, quantitative evaluation: 89%). DeepRadiation application as the DeepRadiation's sescond stage is the framework's eyes. At this stage, two convolutional neural network (CNN) models (DeepLabv3 and MiDaS) were used to perform computer vision tasks on panoramic streetscape images, such as semantic segmentation and depth estimation. The DeepRadiation application stage allows urban designers, architects, and urban policymakers to use the DeepRadiation framework and experience the final output via augmented reality.

Electronic Supplementary Material

Video
bs-16-3-499_ESM.mp4

References

 

Alzubaidi L, Zhang J, Humaidi AJ, et al. (2021). Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data, 8: 53.

 
Baisantry M, Saxena V (2016). Removal of ladder effects by smoothening the integer format elevation layer via development of a QGIS plugin. In: Proceeding of 2016 International Conference on Computing, Communication and Automation (ICCCA), Greater Noida, India.
 

Bamdad K, Cholette ME, Bell J (2020). Building energy optimization using surrogate model and active sampling. Journal of Building Performance Simulation, 13: 760–776.

 

Bennett J (2010). OpenStreetMap. Birmingham, UK: Packt Publishing Ltd.

 

Borji A (2019). Pros and cons of GAN evaluation measures. Computer Vision and Image Understanding, 179: 41–65.

 
Catchpole D, Shkeir N, Smith A (2020). Using Generative Adversarial Networks to create multi-channel images of cells undergoing macropinocytosis. Technical Report, University of Warwick, UK.
 
Chen LC, Papandreou G, Schroff F, et al. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv: 1706.05587.
 

Chokhachian A, Santucci D, Auer T (2017). A human-centered approach to enhance urban resilience, implications and application to improve outdoor comfort in dense urban spaces. Buildings, 7: 113.

 

Cornelis EVK, van Doorn AJ, de Ridder H (2003). Mirror-reflecting a picture of an object: What happens to the shape percept? Perception and Psychophysics, 65: 1110–1125.

 
Du Y, Yu C, Liu J (2009). A study of GIS development based on KML and Google Earth. In: Proceedings of the 5th International Joint Conference on INC, IMS and IDC, Seoul, R. O. Korea.
 
Galanos T, Chronis A (2021). A deep-learning approach to real-time solar radiation prediction. In: The Routledge Companion to Artificial Intelligence in Architecture. New York: Routledge.
 

Gehl J (1987). Life between Buildings (Vol. 23). New York: Van Nostrand Reinhold.

 

Goodfellow I, Bengio Y, Courville A (2016). Deep Learning. Cambridge, MA, USA: MIT Press.

 
Ioffe S, Szegedy C (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of International Conference on Machine Learning.
 
Isola P, Zhu J, Zhou T, et al. (2017). Image-to-image translation with conditional adversarial networks. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.
 
Jiang Z, Jiang N, Wang Y, et al. (2007). Distance measurement in panorama. In: Proceedings of 2007 IEEE International Conference on Image Processing, San Antonio, TX, USA.
 

Khan S, Mohiuddin K (2018). Evaluating the parameters of ArcGIS and QGIS for GIS Applications. International Journal of Advance Research in Science and Engineering, 7: 582–594.

 

Kharvari F (2020). An empirical validation of daylighting tools: Assessing radiance parameters and simulation settings in Ladybug and Honeybee against field measurements. Solar Energy, 207: 1021–1036.

 

Kolečanský Š, Hofierka J, Bogľarský J, et al. (2021). Comparing 2D and 3D solar radiation modeling in urban areas. Energies, 14: 8364.

 

Lee JH, Shin J, Realff MJ (2018). Machine learning: Overview of the recent progresses and implications for the process systems engineering field. Computers and Chemical Engineering, 114: 111–121.

 

Menges A, Ahlquist S (2011). Computational Design Thinking: Computation Design Thinking. Chichester, UK: John Wiley & Sons.

 

Nejat P, Jomehzadeh F, Taheri MM, et al. (2015). A global review of energy consumption, CO2 emissions and policy in the residential sector (with an overview of the top ten CO2 emitting countries). Renewable and Sustainable Energy Reviews, 43: 843–862.

 

Nutkiewicz A, Yang Z, Jain RK (2018). Data-driven Urban Energy Simulation (DUE-S): A framework for integrating engineering simulation and machine learning methods in a multi-scale urban energy modeling workflow. Applied Energy, 225: 1176–1189.

 
Olmos PM, Hitaj B, Gasti P, et al. (2018). What Are GANs Useful For?
 

Pang Y, Lin J, Qin T, et al. (2022). Image-to-image translation: Methods and applications. IEEE Transactions on Multimedia, 24: 3859–3881.

 

Ranftl R, Lasinger K, Hafner D, et al. (2022). Towards robust monocular depth estimation: mixing datasets for zero-shot cross-dataset transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44: 1623–1637.

 

Reinhart CF, Cerezo Davila C (2016). Urban building energy modeling—A review of a nascent field. Building and Environment, 97: 196–202.

 
Sadeghipour Roudsari M, Pak M, Viola A (2013). Ladybug: A parametric environmental plugin for grasshopper to help designers create an environmentally-conscious design. In: Proceedings of the 13th International IBPSA Building Simulation Conference, Chambery, France.
 

Schmidhuber J (2015). Deep learning in neural networks: An overview. Neural Networks, 61: 85–117.

 
Schonfeld E, Schiele B, Khoreva A (2020). A U-Net based discriminator for generative adversarial networks. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.
 
Shi X, Wang C, Wang M, et al. (2020). An innovative approach to determine building window-to-wall ratios for urban energy simulation. In: Proceedings of the 11th Annual Symposium on Simulation for Architecture and Urban Design.
 

Tamke M, Nicholas P, Zwierzycki M (2018). Machine learning for architectural design: Practices and infrastructure. International Journal of Architectural Computing, 16: 123–143.

 

Torralba A, Oliva A (2002). Depth estimation from image structure. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24: 1226–1238.

 

Touloupaki E, Theodosiou T (2017). Performance simulation integrated in parametric 3D modeling as a method for early stage design optimization—A review. Energies, 10: 637.

 
Tsai YHH, Yamada M, Wu D, et al. (2018). Selecting the best in GANs family: A post-selection inference framework. arXiv preprint arXiv. 1802.05411.
 
United Nations (2018). 68% of the world population projected to live in urban areas by 2050, says UN. United Nations Department of Economic and Social Affairs. Available at https://www.un.org. Accessed 24 Jan 2022.
 

Wang Z, Bovik AC, Sheikh HR, et al. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13: 600–612.

 

Warren SG, Eastman RM, Hahn CJ (2007). A survey of changes in cloud cover and cloud types over land from surface observations, 1971–96. Journal of Climate, 20: 717–738.

 
Yazicioglu G, Gürsel Dino I (2021). From streetscape to data—Semantic segmentation for the prediction of outdoor thermal comfort. In: Proceedings of the 39th International Conference on Education and Research in Computer Aided Architectural Design in Europe (eCAADe), Novi Sad, Serbia.
 
Yousif S, Bolojan D (2021). Deep-performance—Incorporating deep learning for automating building performance simulation in generative systems. In: Proceedings of the 26th Conference on Computer Aided Architectural Design Research in Asia (CAADRIA), Hong Kong, China.
 

Zhou C, Zelinka MD, Dessler AE, et al. (2015). The relationship between interannual and long-term cloud feedbacks. Geophysical Research Letters, 42: 10,463–10,469.

Building Simulation
Pages 499-510
Cite this article:
Nakhaee A, Paydar A. DeepRadiation: An intelligent augmented reality platform for predicting urban energy performance just through 360 panoramic streetscape images utilizing various deep learning models. Building Simulation, 2023, 16(3): 499-510. https://doi.org/10.1007/s12273-022-0953-5

519

Views

6

Crossref

6

Web of Science

7

Scopus

1

CSCD

Altmetrics

Received: 04 June 2022
Revised: 05 October 2022
Accepted: 14 October 2022
Published: 21 November 2022
© Tsinghua University Press 2022
Return