Multi-example feature-constrained back-projection method for image super-resolution

Junlei Zhang; Dianguang Gai; Xin Zhang; Xuemei Li

doi:10.1007/s41095-016-0070-4

| Sign up

PDF (5.3 MB)

Cite

EndNote(RIS) BibTeX

Collect

Submit Manuscript

Research Article | Open Access

Multi-example feature-constrained back-projection method for image super-resolution

Junlei Zhang^¹, Dianguang Gai^², Xin Zhang^¹, Xuemei Li^¹()

1 School of Computer Science and Technology, Shandong University, Jinan, 250101, China.

2 Earthquake Administration of Shandong Province, China.

Show Author Information

Abstract

Example-based super-resolution algorithms, which predict unknown high-resolution image information using a relationship model learnt from known high- and low-resolution image pairs, have attracted considerable interest in the field of image processing. In this paper, we propose a multi-example feature-constrained back-projection method for image super-resolution. Firstly, we take advantage of a feature-constrained polynomial interpolation method to enlarge the low-resolution image. Next, we consider low-frequency images of different resolutions to provide an example pair. Then, we use adaptive $k$ NN search to find similar patches in the low-resolution image for every image patch in the high-resolution low-frequency image, leading to a regression model between similar patches to be learnt. The learnt model is applied to the low-resolution high-frequency image to produce high-resolution high-frequency information. An iterative back-projection algorithm is used as the final step to determine the final high-resolution image. Experimental results demonstrate that our method improves the visual quality of the high-resolution image.

Keywords

feature constraints back-projection super-resolution (SR)

References

[1]

Glasner,

; Bagon,

; Irani,

Super-resolution from a single image. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 349-356, 2009.

Crossref

[2]

Park,

S. C.

; Park,

M. K.

; Kang,

M. G.

Super-resolution image reconstruction: A technical overview. IEEE Signal Processing Magazine Vol. 20, No. 3, 21-36, 2003.

Crossref Google Scholar

[3]

Kolte,

; Arora,

Image super-resolution. Available at https://pdfs.semanticscholar.org/20de/2880a4196a733314252a717f1a55f5f0ea64.pdf.

[4]

Hou,

; Andrews,

Cubic splines for image interpolation and digital filtering. IEEE Transactions on Acoustics, Speech, and Signal Processing Vol. 26, No. 6, 508-517, 1978.

Crossref Google Scholar

[5]

McKinley,

; Levine,

Cubic spline interpolation. College of the Redwoods Vol. 45, No. 1, 1049-1060, 1998.

Google Scholar

[6]

Keys,

Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing Vol. 29, No. 6, 1153-1160, 1981.

Crossref Google Scholar

[7]

Wang,

; Gao,

; Zhang,

; Li,

Single-image super-resolution using active-sampling Gaussian process regression. IEEE Transactions on Image Processing Vol. 25, No. 2, 935-948, 2016.

Crossref Google Scholar

[8]

Irani,

; Peleg,

Improving resolution by image registration. CVGIP: Graphical Models and Image Processing Vol. 53, No. 3, 231-239, 1991.

Crossref Google Scholar

[9]

Dong,

; Zhang,

; Shi,

; Wu,

Nonlocal back-projection for adaptive image enlargement. In: Proceedings of the 16th IEEE International Conference on Image Processing, 349-352, 2009.

[10]

Adelson,

E. H.

; Anderson,

C. H.

; Bergen,

J. R.

; Burt,

P. J.

; Ogden,

J. M.

Pyramid methods in image processing. RCA Engineer Vol. 29, No. 6, 33-41, 1984.

Google Scholar

[11]

Bevilacqua,

; Roumy,

; Guillemot,

; Alberi-Morel,

M. L.

Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of British Machine Vision Conference, 135.1-135.10, 2012.

Crossref

[12]

Yang,

C.-Y.

; Huang,

J.-B.

; Yang,

M.-H.

Exploiting self-similarities for single frame super- resolution. In: Computer Vision–ACCV 2010. Kimmel,

; Klette,

; Sugimoto,

Eds. Springer Berlin Heidelberg, 497-510, 2010.

Crossref

[13]

Yang,

; Wright,

; Huang,

T. S.

; Ma,

Image super-resolution via sparse representation. IEEE Transactions on Image Processing Vol. 19, No. 11, 2861-2873, 2010.

Crossref Google Scholar

[14]

Dong,

; Shi,

; Zhang,

; Wu,

Super-resolution with nonlocal regularized sparse representation. In: Proceedings of SPIE7744, Visual Communications and Image Processing, 77440H, 2010.

Crossref

[15]

Yang,

; Wright,

; Huang,

; Ma,

Image super-resolution as sparse representation of raw image patches. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1-8, 2008.

[16]

Zhang,

; Zhang,

; Huang,

T. S.

Efficient sparse representation based image super resolution via dual dictionary learning. In: Proceedings of the IEEE International Conference on Multimedia and Expo, 1-6, 2011.

Crossref

[17]

Zhao,

; Yang,

; Zhang,

; Song,

; Cheng,

; Pan,

Hyperspectral imagery super-resolution by sparse representation and spectral regularization. EURASIP Journal on Advances in Signal Processing Vol. 2011, 87, 2011.

Crossref Google Scholar

[18]

Chang,

; Yeung,

D.-Y.

; Xiong,

Super-resolution through neighbor embedding. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, I, 2004.

[19]

Gao,

; Zhang,

; Tao,

; Li,

Image super-resolution with sparse neighbor embedding. IEEE Transactions on Image Processing Vol. 21, No. 7, 3194-3205, 2012.

Crossref Google Scholar

[20]

Roweis,

S. T.

; Saul,

L. K.

Nonlinear dimensionality reduction by locally linear embedding. Science Vol. 290, No. 5500, 2323-2326, 2000.

Crossref Google Scholar

[21]

BenAbdelkader,

; Cutler,

; Nanda,

; Davis,

EigenGait: Motion-based recognition of people using image self-similarity. In: Audio- and Video-Based Biometric Person Authentication. Bigun,

; Smeraldi,

Eds. Springer Berlin Heidelberg, 284-294, 2001.

Crossref

[22]

Church,

K. W.

; Helfman,

J. I.

Dotplot: A program for exploring self-similarity in millions of lines of text and code. Journal of Computational and Graphical Statistics Vol. 2, No. 2, 153-174, 1993.

Crossref Google Scholar

[23]

Shechtman,

; Irani,

Matching local self-similarities across images and videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1-8, 2007.

Crossref

[24]

Caiming,

; Xin,

; Xuemei,

; Fuhua,

Cubic surface fitting to image with edges as constraints. In: Proceedings of the IEEE International Conference on Image Processing, 1046-1050, 2013.

Crossref

[25]

Chan,

T.-M.

; Zhang,

; Pu,

; Huang,

Neighbor embedding based super-resolution algorithm through edge detection and feature selection. Pattern Recognition Letters Vol. 30, No. 5, 494-502, 2009.

Crossref Google Scholar

[26]

Freedman,

; Fattal,

Image and video upscaling from local self-examples. ACM Transactions on Graphics Vol. 30, No. 2, Article No. 12, 2011.

Crossref Google Scholar

[27]

Dong,

; Zhang,

; Lukac,

; Shi,

Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Transactions on Image Processing Vol. 22, No. 4, 1382-1394, 2013.

Crossref Google Scholar

[28]

Hore,

; Ziou,

Image quality metrics: PSNR vs. SSIM. In: Proceedings of the 20th International Conference on Pattern Recognition, 2366-2369, 2010.

Crossref

Computational Visual Media

Volume 3 Issue 1,
March 2017

Pages 73-82

DOI: 10.1007/s41095-016-0070-4

Cite this article:

Zhang J, Gai D, Zhang X, et al. Multi-example feature-constrained back-projection method for image super-resolution. Computational Visual Media, 2017, 3(1): 73-82. https://doi.org/10.1007/s41095-016-0070-4