AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Computational Visual Media Article
PDF (54 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

Image-based clothes changing system

Computer Science and Engineering, University of Michigan, 2260 Hayward St, Ann Arbor, MI 48109, USA.
Computer Science Department, Stanford University, 353 Serra Mall, Stanford, CA 94305, USA.
School of Engineering and Computer Science, Victoria University of Wellington, Wellington, New Zealand.
TNList, Tsinghua University, Beijing 100084, China.
Show Author Information

Abstract

Current image-editing tools do not match up to the demands of personalized image manipulation, one application of which is changing clothes in user-captured images. Previous work can change single color clothes using parametric human warping methods. In this paper, we propose an image-based clothes changing system, exploiting body factor extraction and content-aware image warping. Image segmentation and mask generation are first applied to the user input. Afterwards, we determine joint positions via a neural network. Then, body shape matching is performed and the shape of the model is warped to the user’s shape. Finally, head swapping is performed to produce realistic virtual results. We also provide a supervision and labeling tool for refinement and further assistance when creating a dataset.

Keywords

clothing try-onimage warpinghuman segmentation

References

[1]
T. Chen,; P. Tan,; L. Q. Ma,; M. M. Cheng,; A. Shamir,; S.-M Hu,. PoseShop: Human image database construction and personalized content synthesis. IEEE Transactions on Visualization and Computer Graphics Vol. 19, No. 5, 824-837, 2013.
Crossref Google Scholar
[2]
S. Zhou,; H. Fu,; L. Liu,; D. Cohen-Or,; X Han,. Parametric reshaping of human bodies in images. ACM Transactions on Graphics Vol. 29, No. 4, Article No. 126, 2010.
Crossref Google Scholar
[3]
V. Ferrari,; M. Marin-Jimenez,; A. Zisserman, Progressive search space reduction for human pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1-8, 2008.
Crossref
[4]
M. Andriluka,; S. Roth,; B. Schiele, Pictorial structures revisited: People detection and articulated pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1014-1021, 2009.
Crossref
[5]
S. Johnson,; M. Everingham, Combining discriminative appearance and segmentation cues for articulated human pose estimation. In: Proceedings of the IEEE 12th International Conference on Computer Vision Workshops, 405-412, 2009.
Crossref
[6]
M. Sun,; S. Savarese, Articulated part-based model for joint object detection and pose estimation. In: Proceedings of the International Conference on Computer Vision, 723-730, 2011.
Crossref
[7]
Y. Tian,; C. L. Zitnick,; S. G. Narasimhan, Exploring the spatial hierarchy of mixture models for human pose estimation. In: Computer Vision-ECCV 2012. A. Fitzgibbon,; S. Lazebnik,; P. Perona,; Y. Sato,; C. Schmid, Eds. Springer-Verlag Berlin Heidelberg, 256-269, 2012.
[8]
Y. Yang,; D. Ramanan, Articulated pose estimation with flexible mixtures-of-parts. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1385-1392, 2011.
Crossref
[9]
V. Vineet,; J. Warrell,; L. Ladicky,; P. H. S. Torr, Human instance segmentation from video using detector-based conditional random fields. In: Proceedings of the 22nd British Machine Vision Conference, 80.1-80.11, 2011.
Crossref
[10]
J. J. Tompson,; A. Jain,; Y. LeCun,; C. Bregler, Joint training of a convolutional network and a graphical model for human pose estimation. In: Proceedings of the Advances in Neural Information Processing Systems 27, 1799-1807, 2014.
[11]
V. Ramakrishna,; D. Munoz,; M. Hebert,; J. A. Bagnell,; Y. Sheikh, Pose machines: Articulated pose estimation via inference machines. In: Computer Vision-ECCV 2014. D. Fleet,; T. Pajdla,; B. Schiele,; T. Tuytelaars, Eds. Springer International Publishing Switzerland, 33-47, 2014.
Crossref
[12]
J. Carreira,; P. Agrawal,; K. Fragkiadaki,; J. Malik, Human pose estimation with iterative error feedback. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4733-4742, 2016.
Crossref
[13]
S.-E. Wei,; V. Ramakrishna,; T. Kanade,; Y. Sheikh, Convolutional pose machines. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4724-4732, 2016.
Crossref
[14]
A. Blake,; C. Rother,; M. Brown,; P. Perez,; P. Torr, Interactive image segmentation using an adaptive GMMRF model. In: Computer Vision-ECCV 2004. T. Pajdla,; J. Matas, Eds. Springer-Verlag Berlin Heidelberg, 428-441, 2004.
Crossref
[15]
L.-C. Chen,; G. Papandreou,; I. Kokkinos,; K. Murphy,; A. L. Yuille, Semantic image segmentation with deep convolutional nets and fully connected CRFS. In: Proceedings of the International Conference on Learning Representations, 2015.
[16]
L. Ladický,; C. Russell,; P. Kohli,; P. H. S. Torr, Associative hierarchical CRFs for object class image segmentation. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 739-746, 2009.
Crossref
[17]
P. Arbeláez,; B. Hariharan,; C. Gu,; S. Gupta,; L. Bourdev,; J. Malik, Semantic segmentation using regions and parts. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3378-3385, 2012.
Crossref
[18]
P. Arbeláez,; M. Maire,; C. Fowlkes,; J Malik,. Contour detection and hierarchical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 5, 898-916, 2011.
Crossref Google Scholar
[19]
P. Arbeláez,; J. Pont-Tuset,; J. T. Barron,; F. Marques,; J. Malik, Multiscale combinatorial grouping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 328-335, 2014.
Crossref
[20]
B. C. Ko,; J.-Y Nam,. Object-of-interest image segmentation based on human attention and semantic region clustering. Journal of the Optical Society of America A Vol. 23, No. 10, 2462-2470, 2006.
Crossref Google Scholar
[21]
M.-M. Cheng,; N. J. Mitra,; X. Huang,; P. H. S Torr,; S.-M Hu,. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 37, No. 3, 569-582, 2015.
Crossref Google Scholar
[22]
M.-M. Cheng,; J. Warrell,; W.-Y. Lin,; S. Zheng,; V. Vineet,; N. Crook, Efficient salient region detection with soft image abstraction. In: Proceedings of the IEEE International Conference on Computer Vision, 1529-1536, 2013.
Crossref
[23]
L.-C. Chen,; Y. Yang,; J. Wang,; W. Xu,; A. L. Yuille, Attention to scale: Scale-aware semantic image segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3640-3649, 2016.
Crossref
[24]
S. Zheng,; M.-M. Cheng,; J. Warrell,; P. Sturgess,; V. Vineet,; C. Rother,; P. H. S. Torr, Dense semantic image segmentation with objects and attributes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3214-3221, 2014.
Crossref
[25]
T. Igarashi,; T. Moscovich,; J. F Hughes,. As-rigid-as-possible shape manipulation. ACM Transactions on Graphics Vol. 24, No. 3, 1134-1141, 2005.
Crossref Google Scholar
[26]
N. Arad,; D Reisfeld,. Image warping using few anchor points and radial functions. Computer Graphics Forum Vol. 14, No. 1, 35-46, 1995.
Crossref Google Scholar
[27]
S. Schaefer,; T. McPhail,; J Warren,. Image deformation using moving least squares. ACM Transactions on Graphics Vol. 25, No. 3, 533-540, 2006.
Crossref Google Scholar
[28]
P. Kaufmann,; O. Wang,; A. Sorkine-Hornung,; O. Sorkine-Hornung,; A. Smolic,; M Gross,. Finite element image warping. Computer Graphics Forum Vol. 32, No. 2, 31-39, 2013.
Crossref Google Scholar
[29]
C. Rother,; V. Kolmogorov,; A Blake,. “GrabCut”: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics Vol. 23, No. 3, 309-314, 2004.
Crossref Google Scholar
[30]
T. Chen,; M.-M. Cheng,; P. Tan,; A. Shamir,; S.-M Hu,. Sketch2Photo: Internet image montage. ACM Transactions on Graphics Vol. 28, No. 5, Article No. 124, 2009.
Crossref Google Scholar
[31]
K. He,; J. Sun,; X. Tang, Guided image filtering. In: Computer Vision-ECCV 2010. K. Daniilidis,; P. Maragos,; N. Paragios, Eds. Springer-Verlag Berlin Heidelberg, 1-14, 2010.
Crossref
[32]
P. Pérez,; M. Gangnet,; A Blake,. Poisson image editing. ACM Transactions on Graphics Vol. 22, No. 3, 313-318, 2003.
Crossref Google Scholar
[33]
E. Shahrian,; D. Rajan, Weighted color and texture sample selection for image matting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 718-725, 2012.
Crossref
[34]
S. Belongie,; J. Malik,; J. Puzicha, Shape context: A new descriptor for shape matching and object recognition. In: Proceedings of the 13th International Conference on Neural Information Processing Systems, 798-804, 2000.
[35]
R. Shilkrot,; D Cohen-Or,; A. Shamir,; L Liu,. Garment personalization via identity transfer. IEEE Computer Graphics and Applications Vol. 33, No. 4, 62-72, 2013.
Crossref Google Scholar
Computational Visual Media
Volume 3 Issue 4,
December 2017
Pages 337-347
DOI: 10.1007/s41095-017-0084-6
Cite this article:
Zheng Z-H, Zhang H-T, Zhang F-L, et al. Image-based clothes changing system. Computational Visual Media, 2017, 3(4): 337-347. https://doi.org/10.1007/s41095-017-0084-6

690

Views

16

Downloads

10

Crossref

N/A

Web of Science

13

Scopus

3

CSCD

Altmetrics
Revised: 17 March 2017
Accepted: 09 April 2017
Published: 08 May 2017
© The Author(s) 2017

This article is published with open access at Springerlink.com

The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
Return