Home Journal of Computer Science and Technology Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Abstract
Keywords
Electronic Supplementary Material
References
Show full outline
Hide outline
Regular Paper

Unsupervised Reconstruction for Gradient-Domain Rendering with Illumination Separation

School of Software, Shandong University, Jinan 250101, China
Show Author Information

Abstract

Gradient-domain rendering methods can render higher-quality images at the same time cost compared with traditional ray tracing rendering methods, and, combined with the neural network, achieve better rendering quality than conventional screened Poisson reconstruction. However, it is still challenging for these methods to keep detailed information, especially in areas with complex indirect illumination and shadows. We propose an unsupervised reconstruction method that separates the direct rendering from the indirect, and feeds them into our unsupervised network with some corresponding auxiliary channels as two separated tasks. In addition, we introduce attention modules into our network which can further improve details. We finally combine the results of the direct and indirect illumination tasks to form the rendering results. Experiments show that our method significantly improves image quality details, especially in scenes with complex conditions.

Keywords

gradient-domain renderingunsupervised networkdeep learning

Electronic Supplementary Material

Download File(s)
JCST-2302-13142-Highlights.pdf (767.8 KB)

References

[1]
Manzi M, Kettunen M, Aittala M, Lehtinen J, Durand F, Zwicker M. Gradient-domain bidirectional path tracing. In Proc. the Eurographics Symposium on Rendering 2015, Jun. 2015, DOI: 10.2312/sre.20151168.
Crossref
[2]

Guo J, Li M, Li Q, Qiang Y, Hu B, Guo Y, Yan L Q. Gradnet: Unsupervised deep screened Poisson reconstruction for gradient-domain rendering. ACM Trans. Graphics, 2019, 38(6): 223. DOI: 10.1145/3355089.3356538.

Crossref Google Scholar
[3]

Kettunen M, Härkönen E, Lehtinen J. Deep convolutional reconstruction for gradient-domain rendering. ACM Trans. Graphics, 2019, 38(4): Article No. 126. DOI: 10.1145/3306346.3323038.

Crossref Google Scholar
[4]

Xu Z, Sun Q, Wang L, Xu Y, Wang B. Unsupervised image reconstruction for gradient-domain volumetric rendering. Computer Graphics Forum, 2020, 39(7): 193–203. DOI: 10.1111/cgf.14137.

Crossref Google Scholar
[5]
Schied C, Kaplanyan A, Wyman C, Patney A, Chaitanya C R A, Burgess J, Liu S, Dachsbacher C, Lefohn A, Salvi M. Spatiotemporal variance-guided filtering: Real-time reconstruction for path-traced global illumination. In Proc. the 2017 High Performance Graphics, Jul. 2017, Article No. 2. DOI: 10.1145/3105762.3105770.
Crossref
[6]

Lehtinen J, Karras T, Laine S, Aittala M, Durand F, Aila T. Gradient-domain metropolis light transport. ACM Trans. Graphics, 2013, 32(4): Article No. 95. DOI: 10.1145/2461912.2461943.

Crossref Google Scholar
[7]

Kettunen M, Manzi M, Aittala M, Lehtinen J, Durand F, Zwicker M. Gradient-domain path tracing. ACM Trans. Graphics, 2015, 34(4): Article No. 123. DOI: 10.1145/2766997.

Crossref Google Scholar
[8]
Bhat P, Curless B, Cohen M, Zitnick C L. Fourier analysis of the 2D screened Poisson equation for gradient domain problems. In Computer Vision–ECCV 2008, Forsyth D, Torr P, Zisserman A (eds.), Springer, 2008, pp.114–128. DOI: 10.1007/978-3-540-88688-4_9.
Crossref
[9]

Rousselle F, Jarosz W, Novák J. Image-space control variates for rendering. ACM Trans. Graphics, 2016, 35(6): Article No. 169. DOI: 10.1145/2980179.2982443.

Crossref Google Scholar
[10]

Manzi M, Vicini D, Zwicker M. Regularizing image reconstruction for gradient-domain rendering with feature patches. Computer Graphics Forum, 2016, 35(2): 263–273. DOI: 10.1111/cgf.12829.

Crossref Google Scholar
[11]

Back J, Yoon S E, Moon B. Feature generation for adaptive gradient-domain path tracing. Computer Graphics Forum, 2018, 37(7): 65–74. DOI: 10.1111/cgf.13548.

Crossref Google Scholar
[12]

Ha S, Oh S, Back J, Yoon S E, Moon B. Gradient outlier removal for gradient-domain path tracing. Computer Graphics Forum, 2019, 38(2): 245–253. DOI: 10.1111/cgf.13634.

Crossref Google Scholar
[13]

He H, Zhao Y, Liu H. Gradient-domain path tracing with gain control strategy. Journal of Computer-Aided Design & Computer Graphics, 2022, 34(2): 294–304. DOI: 10.3724/SP.J.1089.2022.18837. (in Chinese)

Crossref Google Scholar
[14]

Bako S, Vogels T, Mcwilliams B, Meyer M, NováK J, Harvill A, Sen P, Derose T, Rousselle F. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Trans. Graphics, 2017, 36(4): Article No. 97. DOI: 10.1145/3072959.3073708.

Crossref Google Scholar
[15]

Chaitanya C R A, Kaplanyan A S, Schied C, Salvi M, Lefohn A, Nowrouzezahrai D, Aila T. Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. ACM Trans. Graphics, 2017, 36(4): Article No. 98. DOI: 10.1145/3072959.3073601.

Crossref Google Scholar
[16]

Vogels T, Rousselle F, Mcwilliams B, Röthlin G, Harvill A, Adler D, Meyer M, Novák J. Denoising with kernel predictionand asymmetric loss functions. ACM Trans. Graph., 2018, 37(4): Article No. 124. DOI: 10.1145/3197517.3201388.

Crossref Google Scholar
[17]
Suppan C, Chalmers A, Zhao J, Doronin A, Rhee T. Neural screen space rendering of direct illumination. In Proc. the 29th Pacific Conference on Computer Graphics and Applications, Oct. 2021, pp.37–42. DOI: 10.2312/pg.20211385.
[18]

Han K B, Odenthal O G, Kim W J, Yoon S E. Pixel-wise guidance for utilizing auxiliary features in Monte Carlo Denoising. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 2023, 6(1): Article No. 11. DOI: 10.1145/3585505.

Crossref Google Scholar
[19]
Alpay K C, Akyuz A O. Denoising and guided upsampling of Monte Carlo path traced low resolution renderings. In Proc. the 2022 ACM SIGGRAPH Posters, Aug. 2022, Article No. 37. DOI: 10.1145/3532719.3543250.
Crossref
[20]

Zhang X, Ott M, Manzi M, Gross M, Papas M. Automatic feature selection for denoising volumetric renderings. Computer Graphics Forum, 2022, 41(4): 63–77. DOI: 10.1111/cgf.14587.

Crossref Google Scholar
[21]

Firmino A, Frisvad J R, Jensen H W. Progressive denoising of Monte Carlo rendered images. Computer Graphics Forum, 2022, 41(2): 1–11. DOI: 10.1111/cgf.14454.

Crossref Google Scholar
[22]

Currius R R, Assarsson U, Sintorn E. Real-time hair filtering with convolutional neural networks. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 2022, 5(1): Article No. 15. DOI: 10.1145/3522606.

Crossref Google Scholar
[23]

Manzi M, Kettunen M, Durand F, Zwicker M, Lehtinen J. Temporal gradient-domain path tracing. ACM Trans. Graphics, 2016, 35(6): Article No. 246. DOI: 10.1145/2980179.2980256.

Crossref Google Scholar
[24]

Hua B S, Gruson A, Nowrouzezahrai D, Hachisuka T. Gradient-domain photon density estimation. Computer Graphics Forum, 2017, 36(2): 31–38. DOI: 10.1111/cgf.13104.

Crossref Google Scholar
[25]
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Navab N, Hornegger J, Wells W M, Frangi A F (eds.), Springer, 2015, pp.234–241. DOI: 10.1007/978-3-319-24574-4_28.
Crossref
[26]
Woo S, Park J, Lee J Y, Kweon I S. CBAM: Convolutional block attention module. In Computer Vision—ECCV 2018, Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds.), Springer, 2018, pp.3–19. DOI: 10.1007/978-3-030-01234-2_1.
Crossref
[27]
Kingma D P, Ba J. Adam: A method for stochastic optimization. In Proc. the 3rd International Conference on Learning Representations, May 2017.
[28]
He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In Proc. the 2015 IEEE International Conference on Computer Vision (ICCV), Dec. 2015, pp.1026–1034. DOI: 10.1109/ICCV.2015.123.
Crossref
Journal of Computer Science and Technology
Volume 39 Issue 6,
November 2024
Pages 1281-1291
DOI: 10.1007/s11390-024-3142-4
Cite this article:
Ma M-C, Wang L, Xu Y-N, et al. Unsupervised Reconstruction for Gradient-Domain Rendering with Illumination Separation. Journal of Computer Science and Technology, 2024, 39(6): 1281-1291. https://doi.org/10.1007/s11390-024-3142-4
Metrics & Citations  
Article History
Copyright
Return