PDF (24 MB)
Collect
Submit Manuscript
Show Outline
Outline
Abstract
Keywords
References
Show full outline
Hide outline

A Single Image Snow Removal Method for Complex Snowy Scenes

Gang ZHOU()Jingxu RENQiuling LIUJuan CHENZhenhong JIA
Key Laboratory of Signal Detection and Processing, School of Computer Science and Technology, Xinjiang University, Urumqi Xinjiang 830017, China
Show Author Information

Abstract

Snowfall is a complex weather phenomenon that involves multiple scales of snowflakes, complex motion trajectories, and a foggy effect in the long term. However, existing single image snow removal methods and datasets do not take into account these physical properties, which limits the performance of snow removal methods on real snowfall images. To address this issue, we first constructed a snow imaging model that combines snowflakes and fog based on the depth information of the image, and produced a realistic synthetic snowfall image dataset. Then, we designed an end-to-end image restoration network that cascades the defogging and snow removal modules for joint training. The snow removal module adopts a double U-shaped network to solve the problem of multi-scale snowflakes and multi-directional trajectories. Finally, the experimental comparison between our method and other state-of-art methods proves that our method has very good performance in both synthetic datasets and real images, and benefits subsequent visual tasks.

Article ID: 2096-7675(2024)03-0310-09

References

[1]
ZHENG X H, LIAO Y H, GUO W, et al. Single-image-based rain and snow removal using multi-guided filter[C]//International Conference on Neural Information Processing. Berlin, Heidelberg: Springer, 2013: 258-265.
[2]
YU S J, ZHAO Y X, MOU Y, et al. Content-adaptive rain and snow removal algorithms for single image[C]//ZENG Z, LI Y, KING I. International Symposium on Neural Networks. Cham: Springer, 2014: 439-448.
[3]
CHEN X, LI H, LI M Q, et al. Learning a sparse transformer network for effective image deraining[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, BC, Canada. IEEE, 2023: 5896-5905.
[4]
YE Y T, YU C F, CHANG Y, et al. Unsupervised deraining: Where contrastive learning meets self-similarity[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, LA, USA. IEEE, 2022: 5811-5820.
[5]
YANG H, ZHOU D M, ZHAO Q, et al. Image deraining model based on multi-stage dual residual networks[J]. Radio Engineering, 2022, 52(11): 1933-1943. (in Chinese)
[6]
MEI T C, CAO M, YANG H, et al. Two-stage rain removal based on density guidance[J]. Journal of Electronics & Information Technology, 2023, 45(4): 1383-1390. (in Chinese)
[7]
LU B, GAI S. Single image rain removal base on multi scale progressive residual networks[J]. Journal of Image and Graphics, 2022, 27(5): 1537-1553. (in Chinese)
[8]
LIU Y F, JAW D W, HUANG S C, et al. DesnowNet: Context-aware deep network for snow removal[J]. IEEE Transactions on Image Processing, 2018, 27(6): 3064-3073.
[9]
LI P Y, YUN M S, TIAN J D, et al. Stacked dense networks for single-image snow removal[J]. Neurocomputing, 2019, 367: 152-163.
[10]
CHEN W T, FANG H Y, DING J J, et al. JSTASR: Joint size and transparency-aware snow removal algorithm based on modified partial convolution and veiling effect removal[C]//Computer Vision - ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXI. ACM, 2020: 754-770.
[11]
LI Z, ZHANG J, FANG Z J, et al. Single image snow removal via composition generative adversarial networks[J]. IEEE Access, 2019, 7: 25016-25025.
[12]
JAW D W, HUANG S C, KUO S Y. DesnowGAN: An efficient single image snow removal framework using cross-resolution lateral connection and GANs[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(4): 1342-1350.
[13]
ZHANG K H, LI R Q, YU Y J, et al. Deep dense multi-scale network for snow removal using semantic and depth priors[J]. IEEE Transactions on Image Processing, 2021, 30: 7419-7431.
[14]
CHEN W T, FANG H Y, HSIEH C L, et al. All snow removed: Single image desnowing algorithm using hierarchical dual-tree complex wavelet representation and contradict channel loss[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, QC, Canada. IEEE, 2021: 4176-4185.
[15]
VALANARASU J J M, YASARLA R, PATEL V M. TransWeather: Transformer-based restoration of images degraded by adverse weather conditions[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, LA, USA. IEEE, 2022: 2343-2353.
[16]
CHENG B D, LI J C, CHEN Y, et al. Snow mask guided adaptive residual network for image snow removal[J]. Computer Vision and Image Understanding, 2023, 236: 103819.
[17]
GAO T, WEN Y B, ZHANG K H, et al. Towards an effective and efficient transformer for rain-by-snow weather removal[EB/OL]. 2023: arXiv: 2304.02860. https://arxiv.org/abs/2304.02860.pdf.
[18]
LIN J H, JIANG N F, ZHANG Z T, et al. LMQFormer: A laplace-prior-guided mask query transformer for lightweight snow removal[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(11): 6225-6235.
[19]
HU X W, FU C W, ZHU L, et al. Depth-attentional features for single-image rain removal[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA. IEEE, 2019: 8014-8023.
[20]
LI Z Q, SNAVELY N. MegaDepth: Learning single-view depth prediction from Internet photos[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, UT, USA. IEEE, 2018: 2041-2050.
[21]
LI B Y, PENG X L, WANG Z Y, et al. AOD-Net: All-in-one dehazing network[C]//2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy. IEEE, 2017: 4780-4788.
[22]
GAO S H, CHENG M M, ZHAO K, et al. Res2Net: A new multi-scale backbone architecture[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(2): 652-662.
[23]
REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA. IEEE, 2016: 779-788.
Journal of Xinjiang University(Natural Science Edition in Chinese and English)
Pages 310-318
Cite this article:
ZHOU G, REN J, LIU Q, et al. A Single Image Snow Removal Method for Complex Snowy Scenes. Journal of Xinjiang University(Natural Science Edition in Chinese and English), 2024, 41(3): 310-318. https://doi.org/10.13568/j.cnki.651094.651316.2023.08.27.0001
Metrics & Citations  
Article History
Copyright
Return