AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Computational Visual Media Article
PDF (18.7 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

Feature-preserving color pencil drawings from photographs

College of Mathematics and Informatics, South China Agricultural University, Guangzhou, China
School of Computer Science & Engineering, South China University of Technology, Guangzhou, China
School of Data and Computer Science, Sun Yat-sen University, Guangzhou, China
Show Author Information

Graphical Abstract

Abstract

Color pencil drawing is well-loved due to its rich expressiveness. This paper proposes an approach for generating feature-preserving color pencil drawings from photographs. To mimic the tonal style of color pencil drawings, which are much lighter and have relatively lower saturation than photographs, we devise a lightness enhancement mapping and a saturation reduction mapping. The lightness mapping is a monotonically decreasing derivative function, which not only increases lightness but also preserves input photograph features. Color saturation is usually related to lightness, so we suppress the saturation dependent on lightness to yield a harmonious tone. Finally, two extremum operators are provided to generate a foreground-aware outline map in which the colors of the generated contours and the foreground object are consistent. Comprehensive experiments show that color pencil drawings generated by our method surpass existing methods in tone capture and feature preservation.

Keywords

non-photorealistic renderingpencil draw-ingsimage editingfeature preservation

References

[1]
Lewis, D. Pencil Drawing Techniques. Watson-Guptill Publications, 1984.
[2]
Jing, Y. C.; Yang, Y. Z.; Feng, Z. L.; Ye, J. W.; Yu, Y. Z.; Song, M. L. Neural style transfer: A review. IEEE Transactions on Visualization and Computer Graphics Vol. 26, No. 11, 3365–3385, 2020.
Crossref Google Scholar
[3]
Gao, C. Y.; Tang, M. Y.; Liang, X. G.; Su, Z.; Zou, C. Q. PencilArt: A chromatic penciling style generation framework. Computer Graphics Forum Vol. 37, No. 6, 395–409, 2018.
Crossref Google Scholar
[4]
Tong, Z.; Chen, X.; Ni, B.; Wang, X. Sketch generation with drawing process guided by vector flow and grayscale. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, 609–616, 2021.
Crossref
[5]
Li, Y.; Fang, C.; Hertzmann, A.; Shechtman, E.; M. Yang, Im2Pencil: Controllable pencil illustration from photographs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 1525–1534, 2019.
Crossref
[6]
Lu, C.; Xu, L.; Jia, J. Combining sketch and tone for pencil drawing production. In: Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering, 65–73, 2012.
[7]
Chen, D. D.; Yuan, L.; Liao, J.; Yu, N. H.; Hua, G. StyleBank: An explicit representation for neural image style transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2770–2779, 2017.
Crossref
[8]
Véliz, Z. Francisco Pacheco’s comments on painting in oil. Studies in Conservation Vol. 27, No. 2, 49–57, 1982.
Crossref Google Scholar
[9]
Kim, G.; Woo, Y.; Yim, C. Color pencil filter for non-photorealistic rendering applications. In: Proceedings of the 18th IEEE International Symposium on Consumer Electronics, 1–2, 2014.
Crossref
[10]
Cole, F.; Golovinskiy, A.; Limpaecher, A.; Barros, H. S.; Finkelstein, A.; Funkhouser, T.; Rusinkiewicz, S. Where do people draw lines? In: Proceedings of the ACM SIGGRAPH 2008 Papers, Article No. 88, 2008.
Crossref
[11]
Li, S.; Li, K.; Kacher, I.; Taira, Y.; Yanatori, B.; Sato, I. ArtPDGAN: Creating artistic pencil drawing with key map using generative adversarial networks. In: Computational Science – ICCS 2020. Lecture Notes in Computer Science, Vol. 12143. Springer Cham, 285–298, 2020.
Crossref
[12]
Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Wardefarley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, Vol. 2, 2672–2680, 2014.
[13]
He, K. M.; Sun, J.; Tang, X. O. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 6, 1397–1409, 2013.
Crossref Google Scholar
[14]
Ma, S. P.; Ma, H. Q.; Xu, Y. L.; Li, S. A.; Lv, C.; Zhu, M. M. A low-light sensor image enhancement algorithm based on HSI color model. Sensors Vol. 18, No. 10, 3583, 2018.
Crossref Google Scholar
[15]
Chiang, J.; Hsia, C.; Peng, H.; Lien, C. Color image enhancement with saturation adjustment method. Journal of Applied Science and Engineering Vol. 17, No. 4, 341–352, 2014.
Google Scholar
[16]
Zhou, J.; Li, B. X. Automatic generation of pencil-sketch like drawings from personal photos. In: Proceedings of the IEEE International Conference on Multimedia and Expo, 1026–1029, 2005.
[17]
Son, M.; Kang, H.; Lee, Y. J.; Lee, S. Abstract line drawings from 2D images. In: Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, 333–342, 2007.
Crossref
[18]
Bhat, P.; Zitnick, C.; Cohen, M.; Curless, B. GradientShop: A gradient-domain optimization framework for image and video filtering. ACM Transactions on Graphics, Vol. 29, No. 2, Article No. 10, 2010.
Crossref Google Scholar
[19]
Marr, D.; Hildreth, E. Theory of edge detection. Proceedings of the Royal Society of London. Series B. Biological Sciences Vol. 207, No. 1167, 187–217, 1980.
Crossref Google Scholar
[20]
Winnemöller, H.; Olsen, S. C.; Gooch, B. Real-time video abstraction. ACM Transactions on Graphics Vol. 25, No. 3, 1221–1226, 2006.
Crossref Google Scholar
[21]
Kang, H.; Lee, S.; Chui, C. K. Coherent line drawing. In: Proceedings of the 5th International Symposium on Non-Photorealistic Animation and Rendering, 43–50, 2007.
Crossref
[22]
Spicker, M.; Kratt, J.; Arellano, D.; Deussen, O. Depth-aware coherent line drawings. In: Proceedings of the SIGGRAPH Asia 2015 Technical Briefs, Article No. 1, 2015.
Crossref
[23]
Winnemöller, H.; Kyprianidis, J. E.; Olsen, S. C. XDoG: An eXtended difference-of-Gaussians compendium including advanced image stylization. Computers & Graphics Vol. 36, No. 6, 740–753, 2012.
Crossref Google Scholar
[24]
Jin, Y. X.; Li, P.; Sheng, B.; Nie, Y. W.; Kim, J.; Wu, E. H. SRNPD: Spatial rendering network for pencil drawing stylization. Computer Animation and Virtual Worlds Vol. 30, Nos. 3–4, e1890, 2019.
Crossref Google Scholar
[25]
Li, T.; Xie, J. Y.; Niu, H. L.; Hao, S. J. Enhancing pencil drawing patterns via using semantic information. Multimedia Tools and Applications Vol. 81, No. 24, 34245–34262, 2022.
Crossref Google Scholar
[26]
Inoue, N.; Ito, D.; Xu, N.; Yang, J.; Price, B.; T. Yamasaki, Learning to trace: Expressive line drawing generation from photographs. Computer Graphics Forum Vol. 38, No. 7, 69–80, 2019.
Crossref Google Scholar
[27]
Chen, H.; Liu, Z. Q.; Rose, C.; Xu, Y. Q.; Shum,H. Y.; Salesin, D. Example-based composite sketching of human portraits. In: Proceedings of the 3rd International Symposium on Non-Photorealistic Animation and Rendering, 95–153, 2004.
Crossref
[28]
Yi, R.; Liu, Y. J.; Lai, Y. K.; Rosin, P. L. APDrawingGAN: Generating artistic portrait drawings from face photos with hierarchical GANs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10735–10744, 2019.
Crossref
[29]
Yi, R.; Xia, M. F.; Liu, Y. J.; Lai, Y. K.; Rosin, P. L. Line drawings for face portraits from photos using global and local structure based GANs. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 43, No. 10, 3462–3475, 2021.
Crossref Google Scholar
[30]
Liao, J.; Yao, Y.; Yuan, L.; Hua, G.; Kang, S. B. Visual attribute transfer through deep image analogy. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 120, 2017.
Crossref Google Scholar
[31]
Isola, P.; Zhu, J. Y.; Zhou, T. H.; Efros, A. A. Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5967–5976, 2017.
Crossref
[32]
Zhu, J. Y.; Park, T.; Isola, P.; Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, 2242–2251, 2017.
Crossref
[33]
Huang, Z. Y.; Peng, Y. C.; Hibino, T.; Zhao, C. Q.; Xie, H. R.; Fukusato, T.; Miyata, K. DualFace: Two-stage drawing guidance for freehand portrait sketching. Computational Visual Media Vol. 8, No. 1, 63–77, 2022.
Crossref Google Scholar
[34]
Zhou, L.; Yang, Z. H.; Yuan, Q.; Zhou, Z. T.; Hu, D. W. Salient region detection via integrating diffusion-based compactness and local contrast. IEEE Transactions on Image Processing Vol. 24, No. 11, 3308–3320, 2015.
Crossref Google Scholar
[35]
Wang, D.; Zou, C. Q.; Li, G. Q.; Gao, C. Y.; Su, Z.; Tan, P. ℒ0 gradient-preserving color transfer. Computer Graphics Forum Vol. 36, No. 7, 93–103, 2017.
Crossref Google Scholar
[36]
Chan, C.; Durand, F.; Isola, P. Learning to generate line drawings that convey geometry and semantics. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7905–7915, 2022.
Crossref
[37]
sketchKeras. Available at https://github.com/lllyasviel/sketchKeras.
[38]
Sheng, L.; Lin, Z. Y.; Shao, J.; Wang, X. G. Avatar-net: Multi-scale zero-shot style transfer by feature decoration. In: Proceedings of the IEEE/CVF Con-ference on Computer Vision and Pattern Recognition, 8242–8250, 2018.
Crossref
[39]
Xue, Y.; Guo, Y. C.; Zhang, H.; Xu, T.; Zhang, S. H.; Huang, X. L. Deep image synthesis from intuitive user input: A review and perspectives. Computational Visual Media Vol. 8, No. 1, 3–31, 2022.
Crossref Google Scholar
Computational Visual Media
Volume 9 Issue 4,
December 2023
Pages 807-825
DOI: 10.1007/s41095-022-0320-6
Cite this article:
Wang D, Li G, Gao C, et al. Feature-preserving color pencil drawings from photographs. Computational Visual Media, 2023, 9(4): 807-825. https://doi.org/10.1007/s41095-022-0320-6

479

Views

21

Downloads

1

Crossref

2

Web of Science

1

Scopus

0

CSCD

Altmetrics
Received: 10 June 2022
Accepted: 03 October 2022
Published: 30 June 2023
© The Author(s) 2023.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
Return