PDF (9.3 MB)
Collect
Submit Manuscript
Open Access

Information Purification Network for Remote Sensing Image Super-Resolution

College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China
Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
Shanghai Institute of Satellite Engineering, Shanghai 201109, China
Show Author Information

Abstract

Recently, several well-performing deep convolutional neural networks were proposed for remote sensing image super-resolution (SR). However, these methods rarely consider that remote sensing images are corruptible by additional noise, blurring, and other factors. Therefore, to eliminate the interference of these factors, especially the noise, we propose a novel information purification network (IPN) for remote sensing image SR. The proposed information purification block (IPB) can process channel-wise features differently by channel separation and rescale spatial-wise features adaptively through the proposed multi-scale spatial attention mechanism. We further design an information group to explore a more powerful expressive combination of IPBs. Moreover, long and short skip connections can transmit abundant low-frequency information, making IPBs pay more attention to high-frequency information. We mix the images under various degradation models as training data in the training phase. In this way, the network can directly reconstruct various degraded images. Experiments on AID and UC Merced Land-Use datasets under multiple degradation models demonstrate that the proposed IPN performs better than state-of-the-art methods.

References

[1]
L. Zhang and X. Wu, An edge-guided image interpolation algorithm via directional filtering and data fusion, IEEE Trans. Image Process., vol. 15, no. 8, pp. 22262238, 2006.
[2]
K. Zhang, X. Gao, D. Tao, and X. Li, Single image super-resolution with non-local means and steering kernel regression, IEEE Trans. Image Process., vol. 21, no. 11, pp. 45444556, 2012.
[3]
R. Timofte, V. De, and L. Van Gool, Anchored neighborhood regression for fast example-based super-resolution, in Proc. 2013 IEEE Int. Conf. Computer Vision, Sydney, Australia, 2013, pp. 19201927.
[4]
W. He, N. Yokoya, L. Yuan, and Q. Zhao, Remote sensing image reconstruction using tensor ring completion and total variation, IEEE Trans. Geosci. Remote Sens., vol. 57, no. 11, pp. 89989009, 2019.
[5]
Z. Shao, L. Wang, Z. Wang, and J. Deng, Remote sensing image super-resolution using sparse representation and coupled sparse autoencoder, IEEE J. Selected Topics Appl. Earth Obs. Remote Sens., vol. 12, no. 8, pp. 26632674, 2019.
[6]
K. Jiang, Z. Wang, P. Yi, G. Wang, T. Lu, and J. Jiang, Edge-enhanced GAN for remote sensing image superresolution, IEEE Trans. Geosci. Remote Sens., vol. 57, no. 8, pp. 57995812, 2019.
[7]
W. Xu, G. Xu, Y. Wang, X. Sun, D. Lin, and Y. Wu, High quality remote sensing image super-resolution using deep memory connected network, in Proc. 2018 IEEE Int. Geoscience and Remote Sensing Symp., Valencia, Spain, 2018, pp. 88898892.
[8]
C. Dong, C. C. Loy, K. He, and X. Tang, Learning a deep convolutional network for image super-resolution, in Proc. 13th European Conf. Computer Vision, Zurich, Switzerland, 2014, pp. 184199.
[9]
T. Tong, G. Li, X. Liu, and Q. Gao, Image super-resolution using dense skip connections, in Proc. 2017 IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 48094817.
[10]
J. Kim, J. K. Lee, and K. M. Lee, Deeply-recursive convolutional network for image super-resolution, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 16371645.
[11]
S. Liu, R. Gang, C. Li, and R. Song, Adaptive deep residual network for single image super-resolution, Comp. Visual Media, vol. 5, no. 4, pp. 391401, 2019.
[12]
M. Shen, P. Yu, R. Wang, J. Yang, and L. Xue, Image super-resolution via multi-path recursive convolutional network, (in Chinese), Opto-Electron. Eng., vol. 46, no. 11, p. 180489, 2019.
[13]
S. Lei, Z. Shi, and Z. Zou, Super-resolution for remote sensing images via local–global combined network, IEEE Geosci. Remote Sens. Lett., vol. 14, no. 8, pp. 12431247, 2017.
[14]
X. Dong, X. Sun, X. Jia, Z. Xi, L. Gao, and B. Zhang, Remote sensing image super-resolution using novel dense-sampling networks, IEEE Trans. Geosci. Remote Sens., vol. 59, no. 2, pp. 16181633, 2021.
[15]
Z. Pan, W. Ma, J. Guo, and B. Lei, Super-resolution of single remote sensing image based on residual dense backprojection networks, IEEE Trans. Geosci. Remote Sens., vol. 57, no. 10, pp. 79187933, 2019.
[16]
X. Dong, L. Wang, X. Sun, X. Jia, L. Gao, and B. Zhang, Remote sensing image super-resolution using second-order multi-scale networks, IEEE Trans. Geosci. Remote Sens., vol. 59, no. 4, pp. 34733485, 2021.
[17]
S. Wang, T. Zhou, Y. Lu, and H. Di, Contextual transformation network for lightweight remote-sensing image super-resolution, IEEE Trans. Geosci. Remote Sens., vol. 60, p. 5615313, 2022.
[18]
W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 18741883.
[19]
J. Kim, J. K. Lee, and K. M. Lee, Accurate image super-resolution using very deep convolutional networks, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 16461654.
[20]
B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, Enhanced deep residual networks for single image super-resolution, in Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, 2017, pp. 11321140.
[21]
Z. Hui, X. Wang, and X. Gao, Fast and accurate single image super-resolution via information distillation network, in Proc. 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 723731.
[22]
Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, Residual dense network for image super-resolution, in Proc. 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 24722481.
[23]
Y. Qiu, R. Wang, D. Tao, and J. Cheng, Embedded block residual network: A recursive restoration model for single-image super-resolution, in Proc. 2019 IEEE/CVF Int. Conf. Computer Vision (ICCV), Seoul, Republic of Korea, 2019, pp. 41794188.
[24]
J. Liu, W. Zhang, Y. Tang, J. Tang, and G. Wu, Residual feature aggregation network for image super-resolution, in Proc. 2020 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 23562365.
[25]
J. Hu, L. Shen, S. Albanie, G. Sun, and E. Wu, Squeeze-and-excitation networks, IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 8, pp. 20112023, 2020.
[26]
S. Woo, J. Park, J. Y. Lee, and I. S. Kweon, CBAM: Convolutional block attention module, in Proc. 15th European Conf. Computer Vision, Munich, Germany, 2018, pp. 319.
[27]
Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, Image super-resolution using very deep residual channel attention networks, in Proc. 15th European Conf. Computer Vision, Munich, Germany, 2018, pp. 294310.
[28]
T. Dai, J. Cai, Y. Zhang, S. T. Xia, and L. Zhang, Second-order attention network for single image super-resolution, in Proc. 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 1105711066.
[29]
Z. Hui, X. Gao, Y. Yang, and X. Wang, Lightweight image super-resolution with information multi-distillation network, in Proc. 27th ACM Int. Conf. Multimedia, Nice, France, 2019, pp. 20242032.
[30]
L. Courtrai, M. T. Pham, C. Friguet, and S. Lefèvre, Small object detection from remote sensing images with the help of object-focused super-resolution using wasserstein GANs, in Proc. 2020 IEEE Int. Geoscience and Remote Sensing Symp., Waikoloa, HI, USA, 2020, pp. 260263.
[31]
A. Ma, J. Wang, Y. Zhong, and Z. Zheng, FactSeg: Foreground activation-driven small object semantic segmentation in large-scale remote sensing imagery, IEEE Trans. Geosci. Remote Sens., vol. 60, p. 5606216, 2021.
[32]
D. Zhang, J. Shao, X. Li, and H. T. Shen, Remote sensing image super-resolution via mixed high-order attention network, IEEE Trans. Geosci. Remote Sens., vol. 59, no. 6, pp. 51835196, 2021.
[33]
H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, Pyramid scene parsing network, in Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 62306239.
[34]
G. S. Xia, J. Hu, F. Hu, B. Shi, X. Bai, Y. Zhong, L. Zhang, and X. Lu, AID: A benchmark data set for performance evaluation of aerial scene classification, IEEE Trans. Geosci. Remote Sens., vol. 55, no. 7, pp. 39653981, 2017.
[35]
Y. Yang and S. Newsam, Bag-of-visual-words and spatial extensions for land-use classification, in Proc. 18th SIGSPATIAL Int. Conf. Advances in Geographic Information Systems, San Jose, CA, USA, 2010, pp. 270279.
[36]
J. Liu, J. Tang, and G. Wu, Residual feature distillation network for lightweight image super-resolution, in Proc. European Conf. Computer Vision, Glasgow, UK, 2020, pp. 4155.
Tsinghua Science and Technology
Pages 310-321
Cite this article:
Wang Z, Li L, Xing L, et al. Information Purification Network for Remote Sensing Image Super-Resolution. Tsinghua Science and Technology, 2023, 28(2): 310-321. https://doi.org/10.26599/TST.2022.9010002
Metrics & Citations  
Article History
Copyright
Rights and Permissions
Return