AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Building Simulation Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article

Automatic classification of rural building characteristics using deep learning methods on oblique photography

Chengyu Meng1,2, Yuwei Song2, Jiaqi Ji1, Ziyu Jia1,3, Zhengxu Zhou1( ), Peng Gao4, Sunxiangyu Liu5
School of Architecture, Tsinghua University, Beijing, 100084, China
School of Landscape Architecture, Beijing Forestry University, Beijing, 100083, China
China National Engineering Research Center for Human Settlement, China Architecture Design & Research Group, Beijing, China
College of Engineering, Peking University, Beijing, 100871, China
School of Aerospace Engineering, Tsinghua University, Beijing, 100084, China
Show Author Information

Abstract

Rural building is important to the well-being of rural residents, leading to a significant need to carry out extensive surveys and retrofits of many rural buildings. On-site surveys by expert surveyors are currently the main approach, but this traditional method is often expensive and laborious, especially for large-scale survey tasks. Therefore, this study explores an alternative workflow based on deep learning (DL) methods to apply automatic classification of rural building characteristics. Taking four villages in Jizhou District of Tianjin, China as research samples, we tested selected convolutional neural network (CNN) architectures through the establishment of the training database containing 3258 labeled images, under the performance metrics of accuracy, recall and F1 score. The results showed that ResNet50 is the CNN architecture with the best performance, with the comprehensive consideration of overall metrics. Taking accuracy as the performance metric to test the generalization ability of ResNet50, the prediction results for seven building characteristic indicators from low to high are as follows: building function (0.827); building style (0.863); building quality (0.871); building age (0.880); building structure (0.891); abandoned or not (0.959); the number of storeys (0.995). Due to simplicity, accuracy and effectiveness, this workflow is transferable and cost-effective to investigate large-scale villages.

Keywords

convolutional neural networksbuilt environmentrural building surveysworkflowResNet50

References

 
Bengio Y (2009). Learning Deep Architectures for AI. Boston: Now Publishers Inc.https://doi.org/10.1561/9781601982957
Crossref
 

Biljecki F, Ito K (2021). Street view imagery in urban analytics and GIS: A review. Landscape and Urban Planning, 215: 104217.
Crossref Google Scholar
 

Cai J, Li B, Yu W, et al. (2020). Household dampness and their associations with building characteristics and lifestyles: Repeated cross-sectional surveys in 2010 and 2019 in Chongqing, China. Building and Environment, 183: 107172.
Crossref Google Scholar
 
Chen L-C, Barron JT, Papandreou G, et a. (2016). Semantic image segmentation with task-specific edge detection using CNNs and a discriminatively trained domain transform. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.https://doi.org/10.1109/CVPR.2016.492
Crossref
 

Cheng G, Han J (2016). A survey on object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 117: 11–28.
Crossref Google Scholar
 

Cheng G, Yang C, Yao X, et al. (2018). When deep learning meets metric learning: remote sensing image scene classification via learning discriminative CNNs. IEEE Transactions on Geoscience and Remote Sensing, 56: 2811–2821.
Crossref Google Scholar
 
Chollet F (2017). Xception: Deep learning with depthwise separable convolutions. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.https://doi.org/10.1109/CVPR.2017.195
Crossref
 

Dai M, Ward WOC, Meyers G, et al. (2021). Residential building facade segmentation in the urban environment. Building and Environment, 199: 107921.
Crossref Google Scholar
 

Diakogiannis FI, Waldner F, Caccetta P, et al. (2020). ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS Journal of Photogrammetry and Remote Sensing, 162: 94–114.
Crossref Google Scholar
 

Dian R, Li S, Guo A, et al. (2018). Deep hyperspectral image sharpening. IEEE Transactions on Neural Networks and Learning Systems, 29: 5345–5355.
Crossref Google Scholar
 
Dong C, Loy CC, He K, et al. (2014). Learning a deep convolutional network for image super-resolution. In: Proceedings of European Conference on Computer Vision.https://doi.org/10.1007/978-3-319-10593-2_13
Crossref
 
Dong B, Wang X (2016). Comparison deep learning method to traditional methods using for network intrusion detection. In: Proceedings of 2016 8th IEEE International Conference on Communication Software and Networks (ICCSN), Beijing, China.https://doi.org/10.1109/ICCSN.2016.7586590
Crossref
 

Fan C, Yan D, Xiao F, et al. (2021a). Advanced data analytics for enhancing building performances: From data-driven to big data-driven approaches. Building Simulation, 14: 3–24.
Crossref Google Scholar
 

Fan Y, Ding X, Wu J, et al. (2021b). High spatial-resolution classification of urban surfaces using a deep learning method. Building and Environment, 200: 107949.
Crossref Google Scholar
 

Fang X (2020). On the achievements of building a well-off society in an all-round way from the perspective of residents' income and expenditure. People's Daily, 2020-07-27, P. 10. (in Chinese)
Google Scholar
 

Gawrys MR, Carswell AT (2020). Exploring the cost burden of rural rental housing. Journal of Rural Studies, 80: 372–379.
Crossref Google Scholar
 
Gkartzios M, Scott M, Gallent N (2020). Rural housing. In: Kobayashi A (Ed. ), International Encyclopedia of Human Geography, 2nd edn. Amsterdam: the Netherlands.https://doi.org/10.1016/B978-0-08-102295-5.10341-5
Crossref
 

Gong F-Y, Zeng Z-C, Zhang F, et al. (2018). Mapping sky, tree, and building view factors of street canyons in a high-density urban environment. Building and Environment, 134: 155–167.
Crossref Google Scholar
 

Gonzalez D, Rueda-Plata D, Acevedo AB, et al. (2020). Automatic detection of building typology using deep learning methods on street level images. Building and Environment, 177: 106805.
Crossref Google Scholar
 

Guo R, Liu J, Li N, et al. (2018). Pixel-wise classification method for high resolution remote sensing imagery using deep neural networks. ISPRS International Journal of Geo-Information, 7(3): 110.
Crossref Google Scholar
 

Haapio A, Viitaniemi P (2008). A critical review of building environmental assessment tools. Environmental Impact Assessment Review, 28: 469–482.
Crossref Google Scholar
 
He K, Zhang X, Ren S, et al. (2016). Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.https://doi.org/10.1109/CVPR.2016.90
Crossref
 

Himeur Y, Ghanem K, Alsalemi A, et al. (2021). Artificial intelligence based anomaly detection of energy consumption in buildings: A review, current trends and new perspectives. Applied Energy, 287: 116601.
Crossref Google Scholar
 

Höhle J (2021). Automated mapping of buildings through classification of DSM-based ortho-images and cartographic enhancement. International Journal of Applied Earth Observation and Geoinformation, 95: 102237.
Crossref Google Scholar
 

Hu C-B, Zhang F, Gong F-Y, et al. (2020). Classification and mapping of urban canyon geometry using Google Street View images and deep multitask learning. Building and Environment, 167: 106424.
Crossref Google Scholar
 

Hu Q, Zhen L, Mao Y, et al. (2021). Automated building extraction using satellite remote sensing imagery. Automation in Construction, 123: 103509.
Crossref Google Scholar
 

Huang J, Zhang X, Xin Q, et al. (2019). Automatic building extraction from high-resolution aerial images and LiDAR data using gated residual refinement network. ISPRS Journal of Photogrammetry and Remote Sensing, 151: 91–105.
Crossref Google Scholar
 

Jin X, Davis CH (2005). Automated building extraction from high- resolution satellite imagery in urban areas using structural, contextual, and spectral information. EURASIP Journal on Applied Signal Processing, 2005: 2196–2206.
Crossref Google Scholar
 

Jin Y, Yan D, Chong A, et al. (2021). Building occupancy forecasting: A systematical and critical review. Energy and Buildings, 251: 111345.
Crossref Google Scholar
 

Johnston CJ, Andersen RK, Toftum J, et al. (2020). Effect of formaldehyde on ventilation rate and energy demand in Danish homes: Development of emission models and building performance simulation. Building Simulation, 13: 197–212.
Crossref Google Scholar
 
Kamath CN, Bukhari SS, Dengel A (2018). Comparative study between traditional machine learning and deep learning approaches for text classification. In: Proceedings of the ACM Symposium on Document Engineering, Halifax NS, Canada.https://doi.org/10.1145/3209280.3209526
Crossref
 

Kang J, Körner M, Wang Y, et al. (2018). Building instance classification using street view images. ISPRS Journal of Photogrammetry and Remote Sensing, 145: 44–59.
Crossref Google Scholar
 

Kang X, Yan D, An J, et al. (2021). Typical weekly occupancy profiles in non-residential buildings based on mobile positioning data. Energy and Buildings, 250: 111264.
Crossref Google Scholar
 

Kong L, Liu Z, Wu J (2020). A systematic review of big data-based urban sustainability research: State-of-the-science and future directions. Journal of Cleaner Production, 273: 123142.
Crossref Google Scholar
 

Leaman A, Stevenson F, Bordass B (2010). Building evaluation: practice and principles. Building Research & Information, 38: 564–577.
Crossref Google Scholar
 

LeCun Y, Bengio Y, Hinton G (2015). Deep learning. Nature, 521(7553): 436–444.
Crossref Google Scholar
 

Li E, Femiani J, Xu S, et al. (2015). Robust rooftop extraction from visible band images using higher order CRF. IEEE Transactions on Geoscience and Remote Sensing, 53: 4483–4495.
Crossref Google Scholar
 

Li Y, Huang X, Liu H (2017). Unsupervised deep feature learning for urban village detection from high-resolution remote sensing images. Photogrammetric Engineering & Remote Sensing, 83: 567–579.
Crossref Google Scholar
 
Lin T-Y, Goyal P, Girshick R, et al. (2017). Focal loss for dense object detection. In: Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.https://doi.org/10.1109/ICCV.2017.324
Crossref
 

Liu Y (2018). Introduction to land use and rural sustainability in China. Land Use Policy, 74: 1–4.
Crossref Google Scholar
 

Liu Y, Chen X, Wang Z, et al. (2018a). Deep learning for pixel-level image fusion: Recent advances and future prospects. Information Fusion, 42: 158–173.
Crossref Google Scholar
 

Liu Y, Fan B, Wang L, et al. (2018b). Semantic labeling in very high resolution images via a self-cascaded convolutional neural network. ISPRS Journal of Photogrammetry and Remote Sensing, 145: 78–95.
Crossref Google Scholar
 

Liu J, Li T, Xie P, et al. (2020). Urban big data fusion based on deep learning: An overview. Information Fusion, 53: 123–133.
Crossref Google Scholar
 

Lu Z, Im J, Rhee J, et al. (2014). Building type classification using spatial and landscape attributes derived from LiDAR remote sensing data. Landscape and Urban Planning, 130: 134–148.
Crossref Google Scholar
 

Lu S (2021). Regional Architectural language recognition and evaluation based on visual perception: the case of minority housing in Nujiang area. New Architecture, 2021(01): 110–115. (in Chinese)
Google Scholar
 

Lu X, Feng F, Pang Z, et al. (2021a). Extracting typical occupancy schedules from social media (TOSSM) and its integration with building energy modeling. Building Simulation, 14: 25–41.
Crossref Google Scholar
 

Lu Z, Wang T, Guo J, et al. (2021b). Data-driven floor plan understanding in rural residential buildings via deep recognition. Information Sciences, 567: 58–74.
Crossref Google Scholar
 

Lyu P, Yu M, Hu Y (2020). Contradictions in and improvements to urban and rural residents' housing rights in China's urbanization process. Habitat International, 97: 102101.
Crossref Google Scholar
 

Ma L, Liu Y, Zhang X, et al. (2019). Deep learning in remote sensing applications: A meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing, 152: 166–177.
Crossref Google Scholar
 

Mangalathu S, Burton HV (2019). Deep learning-based classification of earthquake-impacted buildings using textual damage descriptions. International Journal of Disaster Risk Reduction, 36: 101111.
Crossref Google Scholar
 

Marcos D, Volpi M, Kellenberger B, et al. (2018). Land cover mapping at very high resolution with rotation equivariant CNNs: Towards small yet accurate models. ISPRS Journal of Photogrammetry and Remote Sensing, 145: 96–107.
Crossref Google Scholar
 

Na H, Choi H, Kim T (2020). Metabolic rate estimation method using image deep learning. Building Simulation, 13: 1077–1093.
Crossref Google Scholar
 

Na R, Shen Z (2021). Assessing cooling energy reduction potentials by retrofitting traditional cavity walls into passively ventilated cavity walls. Building Simulation, 14: 1295–1309.
Crossref Google Scholar
 
National Bureau of Statistics (2020). China Statistical Yearbook. Beijing: Chinese Statistics Press. (in Chinese).
 

Porikli F, Shan S, Snoek C, et al. (2018). Deep learning for visual understanding: part 2. IEEE Signal Processing Magazine, 35(1): 17–19.
Crossref Google Scholar
 

Preiser WF, Nasar JL (2008). Assessing building performance: Its evolution from post-occupancy evaluation. International Journal of Architectural Research, 2(1): 84–99.
Google Scholar
 

Rueda-Plata D, González D, Acevedo AB, et al. (2021). Use of deep learning models in street-level images to classify one-story unreinforced masonry buildings based on roof diaphragms. Building and Environment, 189: 107517.
Crossref Google Scholar
 
Sun Y, Wang X, Tang X (2014). Deep Learning Face Representation from Predicting 10, 000 Classes. In: Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.https://doi.org/10.1109/CVPR.2014.244
Crossref
 
Tan M, Le Q (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. In: Proceedings of International Conference on Machine Learning.
 

Tian W, Zhu C, Sun Y, et al. (2021). Energy characteristics of urban buildings: Assessment by machine learning. Building Simulation, 14: 179–193.
Crossref Google Scholar
 

Venetianer PL, Werblin F, Roska T, et al. (1995). Analogic CNN algorithms for some image compression and restoration tasks. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 42: 278–284.
Crossref Google Scholar
 

Wang C, Yan D, Jiang Y (2011). A novel approach for building occupancy simulation. Building Simulation, 4: 149–167.
Crossref Google Scholar
 

Wang X, Zhao Y, Pourpanah F (2020). Recent advances in deep learning. International Journal of Machine Learning and Cybernetics, 11: 747–750.
Crossref Google Scholar
 

Watts AC, Ambrosia VG, Hinkley EA (2012). Unmanned aircraft systems in remote sensing and scientific research: classification and considerations of use. Remote Sensing, 4: 1671–1692.
Crossref Google Scholar
 
Xie J (2019). Research on key technologies of rural building information extraction based on high resolution remote sensing images. PhD Thesis, Southwest Jiaotong University, China. (in Chinese)
 

Xu X, Li J, Huang X, et al. (2016). Multiple morphological component analysis based decomposition for remote sensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 54: 3083–3102.
Crossref Google Scholar
 

Yan X, Ai T, Yang M, et al. (2019). A graph convolutional neural network for classification of building patterns using spatial vector data. ISPRS Journal of Photogrammetry and Remote Sensing, 150: 259–273.
Crossref Google Scholar
 

Yin L, Wang Z (2016). Measuring visual enclosure for street walkability: Using machine learning algorithms and Google Street View imagery. Applied Geography, 76: 147–153.
Crossref Google Scholar
 
Yong C (2020). Research on method and application of urban built-up area information extraction based on spectral index. Master Thesis, Chongqing University of Posts and Telecommunications, China. (in Chinese)
 

Yu D, Ji S, Liu J, et al. (2021). Automatic 3D building reconstruction from multi-view aerial images with deep learning. ISPRS Journal of Photogrammetry and Remote Sensing, 171: 155–170.
Crossref Google Scholar
 

Yuan Q, Wei Y, Meng X, et al. (2018). A multiscale and multidepth convolutional neural network for remote sensing imagery pan- sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11: 978–989.
Crossref Google Scholar
 

Yuan L, Guo J, Wang Q (2020). Automatic classification of common building materials from 3D terrestrial laser scan data. Automation in Construction, 110: 103017.
Crossref Google Scholar
 
Zampieri A, Charpiat G, Girard N, et al. (2018). Multimodal image alignment through a multiscale chain of neural networks with application to remote sensing. In: Proceedings of the 15th European Conference Computer Vision, Munich, Germany.https://doi.org/10.1007/978-3-030-01270-0_40
Crossref
 

Zhan S, Chong A, Lasternas B (2021). Automated recognition and mapping of building management system (BMS) data points for building energy modeling (BEM). Building Simulation, 14: 43–52.
Crossref Google Scholar
 

Zhang X, Gao P, Zhao K, et al. (2020). Image restoration via deep memory-based latent attention network. IEEE Access, 8: 104728– 104739.
Crossref Google Scholar
 

Zhang F, Fan Z, Kang Y, et al. (2021a). "Perception bias": Deciphering a mismatch between urban crime and perception of safety. Landscape and Urban Planning, 207: 104003.
Crossref Google Scholar
 

Zhang X, Gao P, Liu S, et al. (2021b). Accurate and efficient image super-resolution via global-local adjusting dense network. IEEE Transactions on Multimedia, 23: 1924–1937.
Crossref Google Scholar
 

Zhong B, Xing X, Love P, et al. (2019). Convolutional neural network: Deep learning-based classification of building quality problems. Advanced Engineering Informatics, 40: 46–57.
Crossref Google Scholar
 

Zhou X, Tian S, An J, et al. (2021). Comparison of different machine learning algorithms for predicting air-conditioning operating behavior in open-plan offices. Energy and Buildings, 251: 111347.
Crossref Google Scholar
Building Simulation
Volume 15 Issue 6,
June 2022
Pages 1161-1174
DOI: 10.1007/s12273-021-0872-x
Cite this article:
Meng C, Song Y, Ji J, et al. Automatic classification of rural building characteristics using deep learning methods on oblique photography. Building Simulation, 2022, 15(6): 1161-1174. https://doi.org/10.1007/s12273-021-0872-x

768

Views

27

Crossref

28

Web of Science

27

Scopus

0

CSCD

Altmetrics
Received: 11 July 2021
Revised: 01 December 2021
Accepted: 01 December 2021
Published: 15 December 2021
© Tsinghua University Press and Springer-Verlag GmbH Germany, part of Springer Nature 2021
Return