PDF (29.6 MB)
Collect
Submit Manuscript
Research Article | Open Access

Image smoothing based on global sparsity decomposition and a variable parameter

Shandong University, Jinan 250101, China
Shandong Co-Innovation Center of Future Intelligent Computing, Yantai 264025, China
Digital Media Technology Key Lab of Shandong Province, Jinan 250014, China
Show Author Information

Abstract

Smoothing images, especially with rich texture, is an important problem in computer vision. Obtaining an ideal result is difficult due to complexity, irregularity, and anisotropicity of the texture. Besides, some properties are shared by the texture and the structure in an image. It is a hard compromise to retain structure and simultaneously remove texture. To create an ideal algorithm for image smoothing, we face three problems. For images with rich textures, the smoothing effect should be enhanced. We should overcome inconsistency of smoothing results in different parts of the image. It is necessary to create a method to evaluate the smoothing effect. We apply texture pre-removal based on global sparse decomposition with a variable smoothing parameter to solve the first two problems. A parametric surface constructed by an improved Bessel method is used to determine the smoothing parameter. Three evaluation measures: edge integrity rate, texture removal rate, and gradient value distribution are proposed to cope with the third problem. We use the alternating direction method of multipliers to complete the whole algorithm and obtain the results. Experiments show that our algorithm is better than existing algorithms both visually and quantitatively. We also demonstrate our method’s ability in other applications such as clip-art compression artifact removal and content-aware image manipulation.

References

[1]
Sun, Y. J.; Schaefer, S.; Wang, W. P. Image structure retrieval via L minimization. IEEE Transactions on Visualization and Computer Graphics Vol. 24, No. 7, 2129-2139, 2018.
[2]
Tomasi, C.; Manduchi, R. Bilateral filtering for gray and color images. In: Proceedings of the International Conference on Computer Vision, 839-846, 1998.
[3]
Chen, J. W.; Paris, S.; Durand, F. Real-time edge-aware image processing with the bilateral grid. ACM Transactions on Graphics Vol. 26, No. 3, 103, 2007.
[4]
Weiss, B. Fast median and bilateral filtering. ACM Transactions on Graphics Vol. 25, No. 3, 519-526, 2006.
[5]
Cho, H.; Lee, H.; Kang, H.; Lee, S. Bilateral texture filtering. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 128, 2014.
[6]
Bao, L. C.; Song, Y. B.; Yang, Q. X.; Yuan, H.; Wang, G. Tree filtering: Efficient structure-preserving smoothing with a minimum spanning tree. IEEE Transactions on Image Processing Vol. 23, No. 2, 555-569, 2014.
[7]
Perona, P.; Malik, J. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 12, No. 7, 629-639, 1990.
[8]
He, K. M.; Sun, J.; Tang, X. O. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 6, 1397-1409, 2013.
[9]
Subr, K.; Soler, C.; Durand, F. Edge-preserving multiscale image decomposition based on local extrema. ACM Transactions on Graphics Vol. 28, No. 5, Article No. 147, 2009.
[10]
Farbman, Z.; Fattal, R.; Lischinski, D.; Szeliski, R. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Transactions on Graphics Vol. 27, No. 3, Article No. 67, 2008.
[11]
Lindeberg, T. Scale-space theory: A basic tool for analyzing structures at different scales. Journal of Applied Statistics Vol. 21, Nos. 1-2, 225-270, 1994.
[12]
Mikolajczyk, K.; Schmid, C. An affine invariant interest point detector. In: Proceedings of the 7th European Conference on Computer Vision-Part I, 128-142, 2002.
[13]
Cai, B. L.; Xing, X. F.; Xu, X. M. Edge/structure preserving smoothing via relativity-of-Gaussian. In: Proceedings of the IEEE International Conference on Image Processing, 250-254, 2017.
[14]
Rudin, L. I.; Osher, S.; Fatemi, E. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena Vol. 60, Nos. 1-4, 259-268, 1992.
[15]
Xu, L.; Lu, C. W.; Xu, Y.; Jia, J. Y. Imagesmoothing via L gradient minimization. ACM Transactions on Graphics Vol. 30, No. 6, Article No. 174, 2011.
[16]
Ono, S. L gradient projection. IEEE Transactions on Image Processing Vol. 26, No. 4, 1554-1564, 2017.
[17]
Ono, S. Edge-preserving filtering by projection onto L gradient constraint. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 1492-1496, 2017.
[18]
Xu, L.; Yan, Q.; Xia, Y.; Jia, J. Y. Structure extraction from texture via relative total variation. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 139, 2012.
[19]
Chen, Q. F.; Xu, J.; Koltun, V. Fast image processing with fully-convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, 2516-2525, 2017.
[20]
Xu, L.; Ren, J.; Yan, Q.; Liao, R.; Jia, J. Deep edge-aware filters. In: Proceedings of the 32nd International Conference on Machine Learning, Vol. 37, 1669-1678, 2015.
[21]
Kim, Y.; Ham, B.; Do, M. N.; Sohn, K. Structure-texture image decomposition using deep variational priors. IEEE Transactions on Image Processing Vol. 28, No. 6, 2692-2704, 2019.
[22]
Zhao, M.; Zhang, H. X.; de Sun, J. A novel image retrieval method based on multi-trend structure descriptor. Journal of Visual Communication and Image Representation Vol. 38, 73-81, 2016.
[23]
Zhang, F.; Li, J. J.; Liu, P. Q.; Fan, H. Computing knots by quadratic and cubic polynomial curves. Computational Visual Media Vol. 6, No. 4, 417-430, 2020.
[24]
Liu, X. X.; Zhang, Y. F.; Bao, F. X.; Shao, K.; Sun, Z. Y.; Zhang, C. M. Kernel-blending connection approximated by a neural network for image classification. Computational Visual Media Vol. 6, No. 4, 467-476, 2020.
[25]
Wang, Y. L.; Yang, J. F.; Yin, W. T.; Zhang, Y. A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences Vol. 1, No. 3, 248-272, 2008.
[26]
Gu, S. H.; Xie, Q.; Meng, D. Y.; Zuo, W. M.; Feng, X. C.; Zhang, L. Weighted nuclear norm minimization and its applications to low level vision. International Journal of Computer Vision Vol. 121, No. 2, 183-208, 2017.
[27]
Eckstein, J.; Bertsekas, D. P. On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Mathematical Programming Vol. 55, Nos. 1-3, 293-318, 1992.
[28]
Boyd, S.; Parikh, N.; Chu, E.; Peleato, B.; Eckstein, J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends®in Machine Learning Vol. 3, No. 1, 1-122, 2011.
[29]
Zhang, R.; Kwok, J. Asynchronous distributed ADMM for consensus optimization. In: Proceedings of the 31st International Conference on International Conference on Machine Learning, Vol. 32, II-1701-II-1709, 2014.
[30]
Sun, D. L.; Févotte, C. Alternating direction method of multipliers for non-negative matrix factorization with the beta-divergence. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 6201-6205, 2014.
[31]
Zhang, M. L.; Desrosiers, C. High-quality image restoration using low-rank patch regularization and global structure sparsity. IEEE Transactions on Image Processing Vol. 28, No. 2, 868-879, 2019.
[32]
Ma, G. H.; Zhang, M. L.; Li, X. M.; Zhang, C. M. Image smoothing based on image decomposition and sparse high frequency gradient. Journal of Computer Science and Technology Vol. 33, No. 3, 502-510, 2018.
[33]
Zhu, F. D.; Liang, Z. T.; Jia, X. X.; Zhang, L.; Yu, Y. Z. A benchmark for edge-preserving image smoothing. IEEE Transactions on Image Processing Vol. 28, No. 7,3556-3570, 2019.
[34]
Jian, M. W.; Zhang, W. Y.; Yu, H.; Cui, C. R.; Nie, X. S.; Zhang, H. X.; Yin, Y. Saliency detection based on directional patches extraction and principal local color contrast. Journal of Visual Communication and Image Representation Vol. 57, 1-11, 2018.
Computational Visual Media
Pages 483-497
Cite this article:
Ma X, Li X, Zhou Y, et al. Image smoothing based on global sparsity decomposition and a variable parameter. Computational Visual Media, 2021, 7(4): 483-497. https://doi.org/10.1007/s41095-021-0220-1
Metrics & Citations  
Article History
Copyright
Rights and Permissions
Return