AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Computational Visual Media Article
PDF (10.9 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Review Article | Open Access

Machine learning for digital try-on: Challenges and progress

University of Maryland, College Park, MD 20785, USA
Show Author Information

Abstract

Digital try-on systems for e-commerce have the potential to change people’s lives and provide notable economic benefits. However, their development is limited by practical constraints, such as accurate sizing of the body and realism of demonstrations. We enumerate three open challenges remaining for a complete and easy-to-use try-on system that recent advances in machine learningmake increasingly tractable. For each, we describethe problem, introduce state-of-the-art approaches, and provide future directions.

Keywords

machine learningdigital try-ongarmentmodelinghuman body estimationmaterial modeling

References

[1]
Zheng, Z. H.; Zhang, H. T.; Zhang, F. L.; Mu, T. J. Image-based clothes changing system. Computational Visual Media Vol. 3, No. 4, 337-347, 2017.
Crossref Google Scholar
[2]
Dibra, E.; Jain, H.; Öztireli, C.; Ziegler, R.; Gross, M. HS-Nets: Estimating human body shape from silhouettes with convolutional neural networks. In: Proceedings of the 4th International Conference on 3D Vision, 108-117, 2016.
Crossref
[3]
Bălan, A. O.; Black, M. J. The naked truth: Estimating body shape under clothing. In: Computer Vision - ECCV 2008. Lecture Notes in Computer Science, Vol. 5303. Forsyth, D.; Torr, P.; Zisserman, A. Eds. Springer Berlin, 15-29, 2008.
[4]
Lassner, C.; Romero, J.; Kiefel, M.; Bogo, F.; Black, M. J.; Gehler, P. V. Unite the people: Closing the loop between 3D and 2D human representations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4704-4713, 2017.
Crossref
[5]
Loper, M.; Mahmood, N.; Romero, J.; Pons-Moll, G.; Black, M. J. SMPL: A skinned multi-person linear model. ACM Transactions on Graphics Vol. 34, No. 6, Article No. 248, 2015.
Crossref Google Scholar
[6]
Wei, S.-E.; Ramakrishna, V.; Kanade, T.; Sheikh, Y. Convolutional pose machines. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 4724-4732, 2016.
Crossref
[7]
Cao, Z.; Simon, T.; Wei, S.; Sheikh, Y. Realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1302-1310, 2017.
Crossref
[8]
Mehta, D.; Sridhar, S.; Sotnychenko, O.; Rhodin, H.; Shafiei, M.; Seidel, H.-P.; Xu, W.; Casas, D.; Theobalt, C. VNect: Realtime 3D human pose estimation with a single RGB camera. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 44, 2017.
Crossref Google Scholar
[9]
Alldieck, T.; Magnor, M.; Xu, W.; Theobalt, C.; Pons-Moll, G. Video based reconstruction of 3D people models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8387-8397, 2018.
Crossref
[10]
Kanazawa, A.; Black, M. J.; Jacobs, D. W.; Malik, J. End-to-end recovery of human shape and pose. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7122-7131, 2018.
Crossref
[11]
Varol, G.; Ceylan, D.; Russell, B.; Yang, J.; Yumer, E.; Laptev, I. Bodynet: Volumetric inference of 3D human body shapes. In: Proceedings of the European Conference on Computer Vision, 20-36, 2018.
Crossref
[12]
Zheng, Z.; Yu, T.; Wei, Y.; Dai, Q.; Liu, Y. Deephuman: 3D human reconstruction from a single image. In: Proceedings of the IEEE International Conference on Computer Vision, 7739-7749, 2019.
Crossref
[13]
Saito, S.; Huang, Z.; Natsume, R.; Morishima, S.; Kanazawa, A.; Li, H. PIFu: Pixel-aligned implicit function for high-resolution clothed human digitization. In: Proceedings of the IEEE International Conference on Computer Vision, 2304-2314, 2019.
Crossref
[14]
Xu, Y.; Zhu, S.-C.; Tung, T. Denserac: Joint 3D pose and shape estimation by dense render-and-compare. In: Proceedings of the IEEE International Conference on Computer Vision, 7760-7770, 2019.
Crossref
[15]
Smith, D.; Loper, M.; Hu, X.; Mavroidis, P.; Romero, J. FACSIMILE: Fast and accurate scans from an image in less than a second. In: Proceedings of the IEEE International Conference on Computer Vision, 5329-5338, 2019.
Crossref
[16]
Alldieck, T.; Magnor, M.; Bhatnagar, B. L.; Theobalt, C.; Pons-Moll, G. Learning to reconstruct people in clothing from a single RGB camera. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1175-1186, 2019.
Crossref
[17]
Kolotouros, N.; Pavlakos, G.; Black, M. J.; Daniilidis, K. Learning to reconstruct 3D human pose and shape via modelfitting in the loop. In: Proceedings of the IEEE International Conference on Computer Vision, 2252-2261, 2019.
Crossref
[18]
Liang, J.; Lin, M. C. Shape-aware human pose and shape reconstruction using multi-view images. In: Proceedings of the IEEE International Conference on Computer Vision, 4352-4362, 2019.
Crossref
[19]
Yang, S.; Pan, Z. R.; Amert, T.; Wang, K.; Yu, L. C.; Berg, T.; Lin, M. C. Physics-inspired garment recovery from a single-view image. ACM Transactions on Graphics Vol. 37, No. 5, Article No. 170, 2018.
Crossref Google Scholar
[20]
Yang, S.; Liang, J.; Lin, M. C.; Learning-based cloth material recovery from video. In: Proceedings of the IEEE International Conference on Computer Vision, 4383-4393, 2017.
Crossref
[21]
Qiao, Y. L.; Liang, J. B.; Koltun, V.; Lin, M. C. Scalable differentiable physics for learning and control. arXiv preprint arXiv:2007.02168, 2020.
Google Scholar
[22]
De Avila Belbute-Peres, F.; Smith, K. A.; Allen, K.; Tenenbaum, J.; Kolter, J. Z. End-to-end differentiable physics for learning and control. In: Proceedings of the Advances in Neural Information Processing Systems, 2018.
[23]
Degrave, J.; Hermans, M.; Dambre, J.; Wyffels, F. A differentiable physics engine for deep learning in robotics. Frontiers in Neurorobotics Vol. 13, 6, 2019.
Crossref Google Scholar
[24]
Liang, J.; Lin, M.; Koltun, V. Differentiable cloth simulation for inverse problems. In: Proceedings of the 33rd Conference on Neural Information Processing Systems, 2019.
[25]
Hu, Y.; Liu, J.; Spielberg, A.; Tenenbaum, J. B.; Freeman, W. T.; Wu, J.; Rus, D.; Matusik, W. ChainQueen: A real-time differentiable physical simulator for soft robotics. In: Proceedings of the International Conference on Robotics and Automation, 6265-6271, 2019.
Crossref
[26]
Hu, Y. M.; Anderson, L.; Li, T. M.; Sun, Q.; Carr, N.; Ragan-Kelley, J.; Durand, F. DiffTaichi: Differentiable programming for physical simulation. arXiv preprint arXiv:1910.00935, 2019.
Google Scholar
[27]
Liu, K. X.; Zeng, X. Y.; Bruniaux, P.; Tao, X. Y.; Yao, X. F.; Li, V.; Wang, J. 3D interactive garment pattern-making technology. Computer-Aided Design Vol. 104, 113-124, 2018.
Crossref Google Scholar
[28]
Huang, P.; Yao, J.; Zhao, H. Automatic realistic 3D garment generation based on two images. In: Proceedings of the International Conference on Virtual Reality and Visualization, 250-257, 2016.
Crossref
[29]
Wang, T. Y.; Ceylan, D.; Popović, J.; Mitra, N. J. Learning a shared shape space for multimodal garment design. ACM Transactions on Graphics Vol. 37, No. 6, Article No. 203, 2018.
Crossref Google Scholar
Computational Visual Media
Volume 7 Issue 2,
June 2021
Pages 159-167
DOI: 10.1007/s41095-020-0189-1
Cite this article:
Liang J, Lin MC. Machine learning for digital try-on: Challenges and progress. Computational Visual Media, 2021, 7(2): 159-167. https://doi.org/10.1007/s41095-020-0189-1

883

Views

29

Downloads

6

Crossref

N/A

Web of Science

8

Scopus

2

CSCD

Altmetrics
Received: 24 June 2020
Accepted: 21 July 2020
Published: 23 October 2020
© The Author(s) 2020

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.
Return