AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Computational Visual Media Article
PDF (10 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

Color retargeting: Interactive time-varying color image composition from time-lapse sequences

Department of Electronics and Informatics (ETRO), Vrije Universiteit Brussel (VUB) and iMinds, Pleinlaan 2, Brussels, Belgium.
Department of LISA, unit Image, Université Libre de Bruxelles (ULB), Belgium.
Show Author Information

Abstract

In this paper, we present an interactive static image composition approach, namely color retargeting, to flexibly represent time-varying color editing effect based on time-lapse video sequences. Instead of performing precise image matting or blending techniques, our approach treats the color composition as a pixel-level resampling problem. In order to both satisfy the user’s editing requirements and avoid visual artifacts, we construct a globally optimized interpolation field. This field defines from which input video frames the output pixels should be resampled. Our proposed resampling solution ensures that (i) the global color transition in the output image is as smooth as possible, (ii) the desired colors/objects specified by the user from different video frames are well preserved, and (iii) additional local color transition directions in the image space assigned by the user are also satisfied. Various examples have been shown to demonstrate that our efficient solution enables the user to easily create time-varying color image composition results.

Keywords

image compositiontime-lapse sequencecolor retargetingtime-varying transitionoptimized interpolation field

Electronic Supplementary Material

Video
41095_2015_31_MOESM1_ESM.mp4

References

[1]
Wilkes, S. Day to night. 2015. Available at http://www.stephenwilkes.com/fine-art.
[2]
Setlur, V.; Takagi, S.; Raskar, R.; Gleicher, M.; Gooch, B. Automatic image retargeting. In: Proceedings of the 4th International Conference on Mobile and Ubiquitous Multimedia, 59-68, 2005.
Crossref
[3]
Rubinstein, M.; Shamir, A.; Avidan, S. Improved seam carving for video retargeting. ACM Transactions on Graphics Vol. 27, No. 3, Article No. 16, 2008.
Crossref Google Scholar
[4]
Pérez, P.; Gangnet, M.; Blake, A. Poisson image editing. ACM Transactions on Graphics Vol. 22, No. 3, 313-318, 2003.
Crossref Google Scholar
[5]
Agarwala, A.; Dontcheva, M.; Agrawala, M.; Drucker, S.; Colburn, A.; Curless, B.; Salesin, D.; Cohen, M. Interactive digital photomontage. ACM Transactions on Graphics Vol. 23, No. 3, 294-302, 2004.
Crossref Google Scholar
[6]
Boykov, Y.; Veksler, O.; Zabih, R. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 23, No. 11, 1222-1239, 2001.
Crossref Google Scholar
[7]
Fattal, R.; Lischinski, D.; Werman, M. Gradient domain high dynamic range compression. ACM Transactions on Graphics Vol. 21, No. 3, 249-256, 2002.
Crossref Google Scholar
[8]
Pritch, Y.; Kav-Venaki, E.; Peleg, S. Shift-map image editing. In: Proceedings of IEEE 12th International Conference on Computer Vision, 151-158, 2009.
Crossref
[9]
Farbman, Z.; Hoffer, G.; Lipman, Y.; Cohen-Or, D.; Lischinski, D. Coordinates for instant image cloning. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 67, 2009.
Crossref Google Scholar
[10]
Darabi, S.; Shechtman, E.; Barnes, C.; Goldman, D. B.; Sen, P. Image melding: Combining inconsistent images using patch-based synthesis. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 82, 2012.
Crossref Google Scholar
[11]
Avidan, S.; Shamir, A. Seam carving for content-aware image resizing. ACM Transactions on Graphics Vol. 26, No. 3, Article No. 10, 2007.
Crossref Google Scholar
[12]
Lin, S.-S.; Yeh, I.-C.; Lin, C.-H.; Lee, T.-Y. Patch-based image warping for content-aware retargeting. IEEE Transactions on Multimedia Vol. 15, No. 2, 359-368, 2013.
Crossref Google Scholar
[13]
Zhu, Z.; Martin, R. R.; Hu, S.-M. Panorama completion for street views. Computational Visual Media Vol. 1, No. 1, 49-57, 2015.
Crossref Google Scholar
[14]
Lu, S.-P.; Hanca, J.; Munteanu, A.; Schelkens, P. Depth-based view synthesis using pixel-level image inpainting. In: Proceedings of the 18th International Conference on Digital Signal Processing, 1-6, 2013.
[15]
Levin, A.; Lischinski, D.; Weiss, Y. Colorization using optimization. ACM Transactions on Graphics Vol. 23, No. 3, 689-694, 2004.
Crossref Google Scholar
[16]
Lischinski, D.; Farbman, Z.; Uyttendaele, M.; Szeliski, R. Interactive local adjustment of tonal values. ACM Transactions on Graphics Vol. 25, No. 3, 646-653, 2006.
Crossref Google Scholar
[17]
An, X.; Pellacini, F. AppProp: All-pairs appearance-space edit propagation. ACM Transactions on Graphics Vol. 27, No. 3, Article No. 40, 2008.
Crossref Google Scholar
[18]
Xu, K.; Li, Y.; Ju, T.; Hu, S.-M.; Liu, T.-Q. Efficient affinity-based edit propagation using K-D tree. ACM Transactions on Graphics Vol. 28, No. 5, Article No. 118, 2009.
Crossref Google Scholar
[19]
Farbman, Z.; Fattal, R.; Lischinski, D. Diffusion maps for edge-aware image editing. ACM Transactions on Graphics Vol. 29, No. 6, Article No. 145, 2010.
Crossref Google Scholar
[20]
Chen, X.; Zou, D.; Zhao, Q.; Tan, P. Manifold preserving edit propagation. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 132, 2012.
Crossref Google Scholar
[21]
Xu, L.; Yan, Q.; Jia, J. A sparse control model for image and video editing. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 197, 2013.
Crossref Google Scholar
[22]
Wong, B.-Y.; Shih, K.-T.; Liang, C.-K.; Chen, H. H. Single image realism assessment and recoloring by color compatibility. IEEE Transactions on Multimedia Vol. 14, No. 3, 760-769, 2012.
Crossref Google Scholar
[23]
Ceulemans, B.; Lu, S.-P.; Schelkens, P.; Munteanu, A. Globally optimized multiview video color correction using dense spatio-temporal matching. In: Proceedings of 3DTV-Conference: The True Vision—Capture, Transmission and Display of 3D Video, 1-4, 2015.
Crossref
[24]
Sunkavalli, K.; Matusik, W.; Pfister, H.; Rusinkiewicz, S. Factored time-lapse video. ACM Transactions on Graphics Vol. 26, No. 3, Article No. 101, 2007.
Crossref Google Scholar
[25]
Bennett, E. P.; McMillan, L. Computational time-lapse video. ACM Transactions on Graphics Vol. 26, No. 3, Article No. 102, 2007.
Crossref Google Scholar
[26]
Correa, C. D.; Ma, K.-L. Dynamic video narratives. ACM Transactions on Graphics Vol. 29, No. 4, Article No. 88, 2010.
Crossref Google Scholar
[27]
Rav-Acha, A.; Pritch, Y.; Lischinski, D.; Peleg, S. Dynamosaicing: Mosaicing of dynamic scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 29, No. 10, 1789-1801, 2007.
Crossref Google Scholar
[28]
Lu, S.-P.; Zhang, S.-H.; Wei, J.; Hu, S.-M.; Martin, R. R. Timeline editing of objects in video. IEEE Transactions on Visualization and Computer Graphics Vol. 19, No. 7, 1218-1227, 2013.
Crossref Google Scholar
[29]
Shih, Y.; Paris, S.; Durand, F.; Freeman, W. T. Data-driven hallucination of different times of day from a single outdoor photo. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 200, 2013.
Crossref Google Scholar
[30]
Estrada, F. J. Time-lapse image fusion. In: Proceedings of the 12th International Conference on Computer Vision, Vol. 2, 441-450, 2012.
Crossref
[31]
Tan, J.; Dvorožňák, M.; Sýkora, D.; Gingold, Y. Decomposing time-lapse paintings into layers. ACM Transactions on Graphics Vol. 34, No. 4, Article No. 61, 2015.
Crossref Google Scholar
[32]
Martin-Brualla, R.; Gallup, D.; Seitz, S. M. Time-lapse mining from internet photos. ACM Transactions on Graphics Vol. 34, No. 4, Article No. 62, 2015.
Crossref Google Scholar
[33]
HaCohen, Y.; Shechtman, E.; Goldman, D. B.; Lischinski, D. Optimizing color consistency in photo collections. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 38, 2013.
Crossref Google Scholar
[34]
Lu, S.-P.; Ceulemans, B.; Munteanu, A.; Schelkens, P. Spatiotemporally consistent color and structure optimization for multiview video color correction. IEEE Transactions on Multimedia Vol. 17, No. 5, 577-590, 2015.
Crossref Google Scholar
[35]
Eigen: A C++ linear algebra library. 2015. Available at http://eigen.tuxfamily.org/index. php?title=Main_Page.
Computational Visual Media
Volume 1 Issue 4,
December 2015
Pages 321-330
DOI: 10.1007/s41095-015-0031-3
Cite this article:
Lu S-P, Dauphin G, Lafruit G, et al. Color retargeting: Interactive time-varying color image composition from time-lapse sequences. Computational Visual Media, 2015, 1(4): 321-330. https://doi.org/10.1007/s41095-015-0031-3

736

Views

25

Downloads

11

Crossref

N/A

Web of Science

14

Scopus

0

CSCD

Altmetrics
Revised: 17 November 2015
Accepted: 03 December 2015
Published: 07 January 2016
© The Author(s) 2015

This article is published with open access at Springerlink.com

This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
Return