Multi-example feature-constrained back-projection method for image super-resolution

Junlei Zhang; Dianguang Gai; Xin Zhang; Xuemei Li

doi:10.1007/s41095-016-0070-4

| Sign up

PDF (5.3 MB)

Cite

EndNote(RIS) BibTeX

Collect

Submit Manuscript

Show Outline

Figures (12)

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Tables (2)

Table 1

Table 2

Research Article | Open Access

Multi-example feature-constrained back-projection method for image super-resolution

Junlei Zhang^¹, Dianguang Gai^², Xin Zhang^¹, Xuemei Li^¹()

1 School of Computer Science and Technology, Shandong University, Jinan, 250101, China.

2 Earthquake Administration of Shandong Province, China.

Show Author Information

Abstract

Example-based super-resolution algorithms, which predict unknown high-resolution image information using a relationship model learnt from known high- and low-resolution image pairs, have attracted considerable interest in the field of image processing. In this paper, we propose a multi-example feature-constrained back-projection method for image super-resolution. Firstly, we take advantage of a feature-constrained polynomial interpolation method to enlarge the low-resolution image. Next, we consider low-frequency images of different resolutions to provide an example pair. Then, we use adaptive $k$ NN search to find similar patches in the low-resolution image for every image patch in the high-resolution low-frequency image, leading to a regression model between similar patches to be learnt. The learnt model is applied to the low-resolution high-frequency image to produce high-resolution high-frequency information. An iterative back-projection algorithm is used as the final step to determine the final high-resolution image. Experimental results demonstrate that our method improves the visual quality of the high-resolution image.

Keywords

feature constraints back-projection super-resolution (SR)

References

[1]

Glasner,

; Bagon,

; Irani,

Super-resolution from a single image. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 349-356, 2009.

Crossref

[2]

Park,

S. C.

; Park,

M. K.

; Kang,

M. G.

Super-resolution image reconstruction: A technical overview. IEEE Signal Processing Magazine Vol. 20, No. 3, 21-36, 2003.

Crossref Google Scholar

[3]

Kolte,

; Arora,

Image super-resolution. Available at https://pdfs.semanticscholar.org/20de/2880a4196a733314252a717f1a55f5f0ea64.pdf.

[4]

Hou,

; Andrews,

Cubic splines for image interpolation and digital filtering. IEEE Transactions on Acoustics, Speech, and Signal Processing Vol. 26, No. 6, 508-517, 1978.

Crossref Google Scholar

[5]

McKinley,

; Levine,

Cubic spline interpolation. College of the Redwoods Vol. 45, No. 1, 1049-1060, 1998.

Google Scholar

[6]

Keys,

Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing Vol. 29, No. 6, 1153-1160, 1981.

Crossref Google Scholar

[7]

Wang,

; Gao,

; Zhang,

; Li,

Single-image super-resolution using active-sampling Gaussian process regression. IEEE Transactions on Image Processing Vol. 25, No. 2, 935-948, 2016.

Crossref Google Scholar

[8]

Irani,

; Peleg,

Improving resolution by image registration. CVGIP: Graphical Models and Image Processing Vol. 53, No. 3, 231-239, 1991.

Crossref Google Scholar

[9]

Dong,

; Zhang,

; Shi,

; Wu,

Nonlocal back-projection for adaptive image enlargement. In: Proceedings of the 16th IEEE International Conference on Image Processing, 349-352, 2009.

[10]

Adelson,

E. H.

; Anderson,

C. H.

; Bergen,

J. R.

; Burt,

P. J.

; Ogden,

J. M.

Pyramid methods in image processing. RCA Engineer Vol. 29, No. 6, 33-41, 1984.

Google Scholar

[11]

Bevilacqua,

; Roumy,

; Guillemot,

; Alberi-Morel,

M. L.

Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of British Machine Vision Conference, 135.1-135.10, 2012.

Crossref

[12]

Yang,

C.-Y.

; Huang,

J.-B.

; Yang,

M.-H.

Exploiting self-similarities for single frame super- resolution. In: Computer Vision–ACCV 2010. Kimmel,

; Klette,

; Sugimoto,

Eds. Springer Berlin Heidelberg, 497-510, 2010.

Crossref

[13]

Yang,

; Wright,

; Huang,

T. S.

; Ma,

Image super-resolution via sparse representation. IEEE Transactions on Image Processing Vol. 19, No. 11, 2861-2873, 2010.

Crossref Google Scholar

[14]

Dong,

; Shi,

; Zhang,

; Wu,

Super-resolution with nonlocal regularized sparse representation. In: Proceedings of SPIE7744, Visual Communications and Image Processing, 77440H, 2010.

Crossref

[15]

Yang,

; Wright,

; Huang,

; Ma,

Image super-resolution as sparse representation of raw image patches. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1-8, 2008.

[16]

Zhang,

; Zhang,

; Huang,

T. S.

Efficient sparse representation based image super resolution via dual dictionary learning. In: Proceedings of the IEEE International Conference on Multimedia and Expo, 1-6, 2011.

Crossref

[17]

Zhao,

; Yang,

; Zhang,

; Song,

; Cheng,

; Pan,

Hyperspectral imagery super-resolution by sparse representation and spectral regularization. EURASIP Journal on Advances in Signal Processing Vol. 2011, 87, 2011.

Crossref Google Scholar

[18]

Chang,

; Yeung,

D.-Y.

; Xiong,

Super-resolution through neighbor embedding. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, I, 2004.

[19]

Gao,

; Zhang,

; Tao,

; Li,

Image super-resolution with sparse neighbor embedding. IEEE Transactions on Image Processing Vol. 21, No. 7, 3194-3205, 2012.

Crossref Google Scholar

[20]

Roweis,

S. T.

; Saul,

L. K.

Nonlinear dimensionality reduction by locally linear embedding. Science Vol. 290, No. 5500, 2323-2326, 2000.

Crossref Google Scholar

[21]

BenAbdelkader,

; Cutler,

; Nanda,

; Davis,

EigenGait: Motion-based recognition of people using image self-similarity. In: Audio- and Video-Based Biometric Person Authentication. Bigun,

; Smeraldi,

Eds. Springer Berlin Heidelberg, 284-294, 2001.

Crossref

[22]

Church,

K. W.

; Helfman,

J. I.

Dotplot: A program for exploring self-similarity in millions of lines of text and code. Journal of Computational and Graphical Statistics Vol. 2, No. 2, 153-174, 1993.

Crossref Google Scholar

[23]

Shechtman,

; Irani,

Matching local self-similarities across images and videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1-8, 2007.

Crossref

[24]

Caiming,

; Xin,

; Xuemei,

; Fuhua,

Cubic surface fitting to image with edges as constraints. In: Proceedings of the IEEE International Conference on Image Processing, 1046-1050, 2013.

Crossref

[25]

Chan,

T.-M.

; Zhang,

; Pu,

; Huang,

Neighbor embedding based super-resolution algorithm through edge detection and feature selection. Pattern Recognition Letters Vol. 30, No. 5, 494-502, 2009.

Crossref Google Scholar

[26]

Freedman,

; Fattal,

Image and video upscaling from local self-examples. ACM Transactions on Graphics Vol. 30, No. 2, Article No. 12, 2011.

Crossref Google Scholar

[27]

Dong,

; Zhang,

; Lukac,

; Shi,

Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Transactions on Image Processing Vol. 22, No. 4, 1382-1394, 2013.

Crossref Google Scholar

[28]

Hore,

; Ziou,

Image quality metrics: PSNR vs. SSIM. In: Proceedings of the 20th International Conference on Pattern Recognition, 2366-2369, 2010.

Crossref

Computational Visual Media

Volume 3 Issue 1,
March 2017

Pages 73-82

DOI: 10.1007/s41095-016-0070-4

Cite this article:

Zhang J, Gai D, Zhang X, et al. Multi-example feature-constrained back-projection method for image super-resolution. Computational Visual Media, 2017, 3(1): 73-82. https://doi.org/10.1007/s41095-016-0070-4

Image	CSFI	NeedFS	IUIE	NARM	FCME	Ours
Artwall	26.88	26.38	25.73	26.62	27.43	27.57
Lena	34.98	34.26	33.53	35.31	35.54	35.68
Butterfly	28.27	29.63	26.11	30.27	30.17	30.39
Baby	37.19	36.03	34.61	36.72	37.37	37.55
Zebra	25.38	25.30	24.04	25.63	26.41	26.57
Hat	32.18	32.07	30.88	33.16	33.52	33.58
Pepper	35.25	32.23	33.88	34.80	35.82	36.01
Head	34.88	33.85	33.99	34.37	34.99	35.07
Average	31.89	31.22	30.35	32.11	32.66	32.80

Image	CSFI	NeedFS	IUIE	NARM	FCME	Ours
Artwall	0.81	0.82	0.78	0.81	0.83	0.83
Lena	0.99	0.99	0.96	0.92	0.99	0.99
Butterfly	0.93	0.95	0.91	0.95	0.95	0.96
Baby	0.99	0.99	0.98	0.95	0.99	0.99
Zebra	0.84	0.86	0.82	0.85	0.85	0.86
Hat	0.90	0.92	0.90	0.92	0.92	0.92
Pepper	0.99	0.99	0.98	0.91	0.99	0.99
Head	0.86	0.87	0.85	0.86	0.87	0.87
Average	0.91	0.92	0.90	0.90	0.92	0.92