AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Computational Visual Media Article
PDF (13.7 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

Adaptive sampling and reconstruction for gradient-domain rendering

State Key Lab of CAD&CG, Zhejiang University, Hangzhou 310058, China
College of Transport and Communications, Shanghai Maritime University, Shanghai 201306, China
Show Author Information

Graphical Abstract

Abstract

Gradient-domain rendering estimates finite difference gradients of image intensities and reconstructs the final result by solving a screened Poisson problem, which shows improvements over merely sampling pixel intensities. Adaptive sampling is another orthogonal research area that focuses on distributing samples adaptively in the primal domain. However, adaptive sampling in the gradient domain with low sampling budget has been less explored. Our idea is based on the observation that signals in the gradient domain are sparse, which provides more flexibility for adaptive sampling. We propose a deep-learning-based end-to-end sampling and reconstruction framework in gradient-domain rendering, enabling adaptive sampling gradient and the primal maps simultaneously. We conducted extensive experiments for evaluation and showed that our method produces better reconstruction quality than other methods in the test dataset.

Keywords

gradient-domain renderingadaptive renderingMonte Carlo renderingdeep-learning-based Monte Carlo denoising

Electronic Supplementary Material

Download File(s)
cvm-10-5-885_ESM.zip (159.6 MB)

References

[1]

Kajiya, J. T. The rendering equation. ACM SIGGRAPH Computer Graphics Vol. 20, No. 4, 143–150, 1986.
Crossref Google Scholar
[2]

Zwicker, M.; Jarosz, W.; Lehtinen, J.; Moon, B.; Ramamoorthi, R.; Rousselle, F.; Sen, P.; Soler, C.; Yoon, S. E. Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. Computer Graphics Forum Vol. 34, No. 2, 667–681, 2015.
Crossref Google Scholar
[3]

Kuznetsov, A.; Kalantari, N. K.; Ramamoorthi, R. Deep adaptive sampling for low sample count rendering. Computer Graphics Forum Vol. 37, No. 4, 35–44, 2018.
Crossref Google Scholar
[4]

Hua, B. S.; Gruson, A.; Petitjean, V.; Zwicker, M.; Nowrouzezahrai, D.; Eisemann, E.; Hachisuka, T. A survey on gradient-domain rendering. Computer Graphics Forum Vol. 38, No. 2, 455–472, 2019.
Crossref Google Scholar
[5]

Lehtinen, J.; Karras, T.; Laine, S.; Aittala, M.; Durand, F.; Aila, T. Gradient-domain metropolis light transport. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 95, 2013.
Crossref Google Scholar
[6]

Kettunen, M.; Manzi, M.; Aittala, M.; Lehtinen, J.; Durand, F.; Zwicker, M. Gradient-domain path tracing. ACM Transactions on Graphics Vol. 34, No. 4, Article No. 123, 2015.
Crossref Google Scholar
[7]

Hua, B. S.; Gruson, A.; Nowrouzezahrai, D.; Hachisuka, T. Gradient-domain photon density estimation. Computer Graphics Forum Vol. 36, No. 2, 31–38, 2017.
Crossref Google Scholar
[8]
Manzi, M.; Kettunen, M.; Aittala, M.; Lehtinen, J.; Durand, F.; Zwicker, M. Gradient-domain bidirectional path tracing. In: Proceedings of the Eurographics Symposium on Rendering: Experimental Ideas & Implementations, 65–74, 2015.
Crossref
[9]
Sun, W.; Sun, X.; Carr, N.; Nowrouzezahrai, D.; Ramamoorthi, R. Gradient-domain vertex connection and merging. In: Proceedings of the Eurographics Symposium on Rendering: Experimental Ideas & Implementations, 83–92, 2017.
[10]

Manzi, M.; Kettunen, M.; Durand, F.; Zwicker, M.; Lehtinen, J. Temporal gradient-domain path tracing. ACM Transactions on Graphics Vol. 35, No. 6, Article No. 246, 2016.
Crossref Google Scholar
[11]

Huo, Y.; Yoon, S. E. A survey on deep learning-based Monte Carlo denoising. Computational Visual Media Vol. 7, No. 2, 169–185, 2021.
Crossref Google Scholar
[12]

Kettunen, M.; Härkönen, E.; Lehtinen, J. Deep convolutional reconstruction for gradient-domain rendering. ACM Transactions on Graphics Vol. 38, No. 4, Article No. 126, 2019.
Crossref Google Scholar
[13]

Guo, J.; Li, M.; Li, Q.; Qiang, Y.; Hu, B.; Guo, Y.; Yan, L. Q. GradNet: Unsupervised deep screened Poisson reconstruction for gradient-domain rendering. ACM Transactions on Graphics Vol. 38, No. 6, Article No. 223, 2019.
Crossref Google Scholar
[14]

Hasselgren, J.; Munkberg, J.; Salvi, M.; Patney, A.; Lefohn, A. Neural temporal adaptive sampling and denoising. Computer Graphics Forum Vol. 39, No. 2, 147–155, 2020.
Crossref Google Scholar
[15]
Veach, E. Robust Monte Carlo methods for light transport simulation. Ph.D. Dissertation. Stanford University, 1997.
[16]

Hachisuka, T.; Jensen, H. W. Stochastic progressive photon mapping. ACM Transactions on Graphics Vol. 28, No. 5, Article No. 141, 2009.
Crossref Google Scholar
[17]
Georgiev, I. Implementing vertex connection and merging. Technical Report. Saarland University, 2012. Available at https://iliyan.com/publications/ImplementingVCM/ImplementingVCM_TechRep2012_rev2.pdf
[18]

Bauszat, P.; Petitjean, V.; Eisemann, E. Gradient-domain path reusing. ACM Transactions on Graphics Vol. 36, No. 6, Article No. 229, 2017.
Crossref Google Scholar
[19]

Gruson, A.; Hua, B. S.; Vibert, N.; Nowrouzezahrai, D.; Hachisuka, T. Gradient-domain volumetric photon density estimation. ACM Transactions on Graphics Vol. 37, No. 4, Article No. 82, 2018.
Crossref Google Scholar
[20]

Jarosz, W.; Zwicker, M.; Jensen, H. W. The beam radiance estimate for volumetric photon mapping. Computer Graphics Forum Vol. 27, No. 2, 557–566, 2008.
Crossref Google Scholar
[21]

Jarosz, W.; Nowrouzezahrai, D.; Thomas, R.; Sloan, P. P.; Zwicker, M. Progressive photon beams. ACM Transactions on Graphics Vol. 30, No. 5, 1–12, 2011.
Crossref Google Scholar
[22]

Bitterli, B.; Jarosz, W. Beyond points and beams: Higher-dimensional photon samples for volumetric light transport. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 112, 2017.
Crossref Google Scholar
[23]

Petitjean, V.; Bauszat, P.; Eisemann, E. Spectral gradient sampling for path tracing. Computer Graphics Forum Vol. 37, No. 4, 45–53, 2018.

Crossref Google Scholar
[24]

Back, J.; Yoon, S. E.; Moon, B. Feature generation for adaptive gradient-domain path tracing. Computer Graphics Forum Vol. 37, No. 7, 65–74, 2018.
Crossref Google Scholar
[25]

Moon, B.; McDonagh, S.; Mitchell, K.; Gross, M. Adaptive polynomial rendering. ACM Transactions on Graphic Vol. 35, No. 4, Article No. 40, 2016.
Crossref Google Scholar
[26]

Chaitanya, C. R. A.; Kaplanyan, A. S.; Schied, C.; Salvi, M.; Lefohn, A.; Nowrouzezahrai, D.; Aila, T. Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 98, 2017.
Crossref Google Scholar
[27]

Bako, S.; Vogels, T.; McWilliams, B.; Meyer, M.; NováK, J.; Harvill, A.; Sen, P.; Derose, T.; Rousselle, F. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 97, 2017.
Crossref Google Scholar
[28]

Vogels, T.; Rousselle, F.; McWilliams, B.; Röthlin, G.; Harvill, A.; Adler, D.; Meyer, M.; Novák, J. Denoising with kernel prediction and asymmetric loss functions. ACM Transactions on Graphics Vol. 37, No. 4, Article No. 124, 2018.
Crossref Google Scholar
[29]

Xu, B.; Zhang, J.; Wang, R.; Xu, K.; Yang, Y. L.; Li, C.; Tang, R. Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation. ACM Transactions on Graphics Vol. 38, No. 6, Article No. 224, 2019.
Crossref Google Scholar
[30]
Goodfellow, I. J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y.; Haloui, I.; Gupta, J. S.; et al. Generative adversarial networks. arXiv preprint arXiv:1406.2661, 2014.
[31]

Lin, W.; Wang, B.; Wang, L.; Holzschuch, N. A detail preserving neural network model for Monte Carlo denoising. Computational Visual Media Vol. 6, No. 2, 157–168, 2020.
Crossref Google Scholar
[32]

Lin, W.; Wang, B.; Yang, J.; Wang, L.; Yan, L. Q. Path-based Monte Carlo denoising using a three-scale neural network. Computer Graphics Forum Vol. 40, No. 1, 369–381, 2021.
Crossref Google Scholar
[33]

Kettunen, M.; Härkönen, E.; Lehtinen, J. Deep convolutional reconstruction for gradient-domain rendering. ACM Transactions on Graphics Vol. 38, No. 4, Article No. 126, 2019.
Crossref Google Scholar
[34]
Huang, G.; Liu, Z.; van der Maaten, L.; Weinberger, K. Q. Densely connected convolutional networks. arXiv preprint arXiv:1608.06993, 2016.
Crossref
[35]

Xu, Z.; Sun, Q.; Wang, L.; Xu, Y.; Wang, B. Unsupervised image reconstruction for gradient-domain volumetric rendering. Computer Graphics Forum Vol. 39, No. 7, 193–203, 2020.
Crossref Google Scholar
[36]
Clevert, D. A.; Unterthiner, T.; Hochreiter, S. Fast and accurate deep network learning by exponential linear units (ELUs). arXiv preprint arXiv:1511.07289, 2015.
[37]
Kettunen, M.; Härkönen, E.; Lehtinen, J. E-LPIPS: Robust perceptual image similarity via random transformation ensembles. arXiv preprint arXiv:1906. 03973, 2019.
[38]
Zhang, R.; Isola, P.; Efros, A. A.; Shechtman, E.; Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 586–595, 2018.
Crossref
[39]

Reinhard, E.; Stark, M.; Shirley, P.; Ferwerda, J. Photographic tone reproduction for digital images. ACM Transactions on Graphics Vol. 21, No. 3, 267–276, 2002.
Crossref Google Scholar
[40]
Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. PyTorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703, 2019.
[41]
Bitterli, B. Rendering resources. 2016. Available at https://benedikt-bitterli.me/resources/
[42]

Manzi, M.; Rousselle, F.; Kettunen, M.; Lehtinen, J.; Zwicker, M. Improved sampling for gradient-domain metropolis light transport. ACM Transactions on Graphics Vol. 33, No. 6, Article No. 178, 2014.
Crossref Google Scholar
[43]

Chaitanya, C. R. A.; Kaplanyan, A. S.; Schied, C.; Salvi, M.; Lefohn, A.; Nowrouzezahrai, D.; Aila, T. Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 98, 2017.
Crossref Google Scholar
[44]
Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K. Q. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2261–2269, 2017.
Crossref
Computational Visual Media
Volume 10 Issue 5,
October 2024
Pages 885-902
DOI: 10.1007/s41095-023-0361-5
Cite this article:
Liang Y, Liu T, Huo Y, et al. Adaptive sampling and reconstruction for gradient-domain rendering. Computational Visual Media, 2024, 10(5): 885-902. https://doi.org/10.1007/s41095-023-0361-5

109

Views

5

Downloads

0

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 24 January 2023
Accepted: 08 June 2023
Published: 10 October 2024
© The Author(s) 2024.

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
Return