AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Computational Visual Media Article
PDF (4.5 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

Progressive edge-sensing dynamic scene deblurring

School of Information and Electronic Engineering, Shandong Technology and Business University, Yantai 264005, China
School of Computer Science and Technology, Shandong Technology and Business University, Yantai 264005, China
Institute of ZhongKe Network Technology, Yantai 264005, China
Show Author Information

Graphical Abstract

Abstract

Deblurring images of dynamic scenes is a challenging task because blurring occurs due to a combination of many factors. In recent years, the use of multi-scale pyramid methods to recover high-resolution sharp images has been extensively studied. We have made improvements to the lack of detail recovery in the cascade structure through a network using progressive integration of data streams. Our new multi-scale structure and edge feature perception design deals with changes in blurring at different spatial scales and enhances the sensitivity of the network to blurred edges. The coarse-to-fine architecture restores the image structure, first performing global adjustments, and then performing local refinement. In this way, not only is global correlation considered, but also residual information is used to significantly improve image restoration and enhance texture details. Experimental results show quantitative and qualitative improvements over existing methods.

Keywords

multi-scaleimage deblurringdynamic scenesedge features

References

[1]
Gupta, A.; Joshi, N.; Lawrence Zitnick, C.; Cohen, M.; Curless, B. Single image deblurring using motion density functions. In: Computer Vision - ECCV 2010. Lecture Notes in Computer Science, Vol. 6311. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer Berlin Heidelberg, 171-184, 2010.
Crossref
[2]
Harmeling, S.; Hirsch, M.; Schölkopf, B. Space-variant single-image blind deconvolution for removing camera shake. In: Proceedings of the Advances in Neural Information Processing Systems 23, 829-837, 2010,
[3]
Hirsch, M.; Schuler, C. J.; Harmeling, S.; Schölkopf, B. Fast removal of non-uniform camera shake. In: Proceedings of the International Conference on Computer Vision, 463-470, 2011.
Crossref
[4]
Pan, J. S.; Sun, D. Q.; Pfister, H.; Yang, M. H. Blind image deblurring using dark channel prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1628-1636, 2016.
Crossref
[5]
Xu, L.; Zheng, S. C.; Jia, J. Y. Unnatural L0 sparse representation for natural image deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1107-1114, 2013.
Crossref
[6]
Chan, T. F.; Wong, C. K. Total variation blind deconvolution. IEEE Transactions on Image Processing Vol. 7, No. 3, 370-375, 1998.
Crossref Google Scholar
[7]
Cho, S.; Lee, S. Fast motion deblurring. ACM Transactions on Graphics Vol. 28, No. 5, 1-8, 2009.
Crossref Google Scholar
[8]
Pan, J. S.; Hu, Z.; Su, Z. X.; Yang, M. H. Deblurring text images via L0-regularized intensity and gradient prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2901-2908, 2014.
Crossref
[9]
Couzinié-Devy, F.; Sun, J.; Alahari, K.; Ponce, J. Learning to estimate and remove non-uniform image blur. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1075-1082, 2013.
Crossref
[10]
Kim, T. H.; Ahn, B.; Lee, K. M. Dynamic scene deblurring. In: Proceedings of the IEEE International Conference on Computer Vision, 3160-3167, 2013.
Crossref
[11]
Li, J. J.; Li, G. H.; Fan, H. Image dehazing using residual-based deep CNN. IEEE Access Vol. 6, 26831-26842, 2018.
Crossref Google Scholar
[12]
Li, J. J.; Feng, X. M.; Hua, Z. Low-light image enhancement via progressive-recursive network. IEEE Transactions on Circuits and Systems for Video Technology Vol. 31, No. 11, 4227-4240, 2021.
Crossref Google Scholar
[13]
Zhang, T. L.; Li, J. J.; Hua, Z. Iterative multi-scale residual network for deblurring. IET Image Processing Vol. 15, No. 8, 1583-1595, 2021.
Crossref Google Scholar
[14]
Schuler, C. J.; Burger, H. C.; Harmeling, S.; Schölkopf, B. A machine learning approach for non-blind image deconvolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1067-1074, 2013.
Crossref
[15]
Xu, L.; Ren, J. S. J.; Liu, C.; Jia, J. Y. Deep convolutional neural network for image deconvolution. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, Vol. 1, 1790-1798, 2014.
[16]
Schuler, C. J.; Hirsch, M.; Harmeling, S.; Schölkopf, B. Learning to deblur. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 38, No. 7, 1439-1451, 2016.
Crossref Google Scholar
[17]
Tai, Y. W.; Chen, X. G.; Kim, S.; Kim, S. J.; Li, F.; Yang, J.; Yu, J.; Matsushita, Y.; Brown, M. S. Nonlinear camera response functions and image deblurring: Theoretical analysis and practice. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 10, 2498-2512, 2013.
Crossref Google Scholar
[18]
Deng, J.; Dong, W.; Socher, R.; Li, L. J.; Kai, L.; Li, F. F. ImageNet: A large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 248-255, 2009.
Crossref
[19]
Chakrabarti, A. A neural approach to blind motion deblurring. In: Computer Vision - ECCV 2016. Lecture Notes in Computer Science, Vol. 9907. Leibe, B.; Matas, J.; Sebe, N.; Welling, M. Eds. Springer Cham, 221-235, 2016.
Crossref
[20]
Kupyn, O.; Budzan, V.; Mykhailych, M.; Mishkin, D.; Matas, J. DeblurGAN: Blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8183-8192, 2018.
Crossref
[21]
Kupyn, O.; Martyniuk, T.; Wu, J. R.; Wang, Z. Y. DeblurGAN-v2: Deblurring (orders-of-magnitude) faster and better. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 8877-8886, 2019.
Crossref
[22]
Gao, H. Y.; Tao, X.; Shen, X. Y.; Jia, J. Y. Dynamic scene deblurring with parameter selective sharing and nested skip connections. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3843-3851, 2019.
Crossref
[23]
Nah, S.; Kim, T. H.; Lee, K. M. Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 257-265, 2017.
Crossref
[24]
Tao, X.; Gao, H. Y.; Shen, X. Y.; Wang, J.; Jia, J. Y. Scale-recurrent network for deep image deblurring. In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8174-8182, 2018.
Crossref
[25]
Park, D.; Kang, D. U.; Kim, J.; Chun, S. Y. Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12351. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 327-343, 2020.
[26]
Fergus, R.; Singh, B.; Hertzmann, A.; Roweis, S. T.; Freeman, W. T. Removing camera shake from a single photograph. In: Proceedings of the ACM SIGGRAPH 2006 Papers, 787-794, 2006.
Crossref
[27]
Zhang, H. G.; Dai, Y. C.; Li, H. D.; Koniusz, P. Deep stacked hierarchical multi-patch network for image deblurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5971-5979, 2019.
Crossref
[28]
Mao, X. J.; Shen, C. H.; Yang, Y. B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. arXiv preprint arXiv:1603.09056, 2016.
Google Scholar
[29]
Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015. Lecture Notes in Computer Science, Vol. 9351. Navab, N.; Hornegger, J.; Wells, W.; Frangi, A. Eds. Springer Cham, 234-241, 2015.
Crossref
[30]
Ye, M. Y.; Lyu, D.; Chen, G. S. Scale-iterative upscaling network for image deblurring. IEEE Access Vol. 8, 18316-18325, 2020.
Crossref Google Scholar
[31]
Lim, B.; Son, S.; Kim, H.; Nah, S.; Lee, K. M. Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 1132-1140, 2017.
Crossref
[32]
Mirza, M.; Osindero, S. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
Google Scholar
[33]
Wang, Z.; Bovik, A. C.; Sheikh, H. R.; Simoncelli, E. P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing Vol. 13, No. 4, 600-612, 2004.
Crossref Google Scholar
[34]
Levin, A.; Weiss, Y.; Durand, F.; Freeman, W. T. Understanding and evaluating blind deconvolution algorithms. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1964-1971, 2009.
Crossref
[35]
Sun, L. B.; Cho, S.; Wang, J.; Hays, J. Edge-based blur kernel estimation using patch priors. In: Proceedings of the IEEE International Conference on Computational Photography, 1-8, 2013.
[36]
Köhler, R.; Hirsch, M.; Mohler, B.; Schölkopf, B.; Harmeling, S. Recording and playback of camera shake: Benchmarking blind deconvolution with a real-world database. In: Computer Vision - ECCV 2012. Lecture Notes in Computer Science, Vol. 7578. Fitzgibbon, A.; Lazebnik, S.; Perona, P.; Sato, Y.; Schmid, C. Eds. Springer Berlin Heidelberg, 27-40, 2012.
[37]
Kingma, D. P.; Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Google Scholar
[38]
Valberg, A. Light Vision Color. John Wiley and Sons, 2005.
[39]
Bai, Y. C.; Cheung, G.; Liu, X. M.; Gao, W. Graph-based blind image deblurring from a single photograph. IEEE Transactions on Image Processing Vol. 28, No. 3, 1404-1418, 2019.
Crossref Google Scholar
[40]
Liu, J.; Yan, M.; Zeng, T. Surface-aware blind image deblurring. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 43, No. 3, 1041-1055, 2021.
Crossref Google Scholar
[41]
Whyte, O.; Sivic, J.; Zisserman, A.; Ponce, J. Non-uniform deblurring for Shaken images. International Journal of Computer Vision Vol. 98, No. 2, 168-186, 2012.
Crossref Google Scholar
Computational Visual Media
Volume 8 Issue 3,
September 2022
Pages 495-508
DOI: 10.1007/s41095-021-0246-4
Cite this article:
Zhang T, Li J, Fan H. Progressive edge-sensing dynamic scene deblurring. Computational Visual Media, 2022, 8(3): 495-508. https://doi.org/10.1007/s41095-021-0246-4

828

Views

25

Downloads

14

Crossref

14

Web of Science

17

Scopus

0

CSCD

Altmetrics
Received: 04 June 2021
Accepted: 13 June 2021
Published: 06 January 2022
© The Author(s) 2021.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.
Return