AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Computer Science and Technology Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Regular Paper

DEMC: A Deep Dual-Encoder Network for Denoising Monte Carlo Rendering

Department of Computer Science and Technology, Dalian University of Technology, Dalian 116024, China
Department of Computer Science, The University of Hong Kong, Pokfulam, Hong Kong, China
Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong, China
School of Creative Media, City University of Hong Kong, Kowloon, Hong Kong, China

Recommended by CVM 2019

Show Author Information

Abstract

In this paper, we present DEMC, a deep dual-encoder network to remove Monte Carlo noise efficiently while preserving details. Denoising Monte Carlo rendering is different from natural image denoising since inexpensive by-products (feature buffers) can be extracted in the rendering stage. Most of them are noise-free and can provide sufficient details for image reconstruction. However, these feature buffers also contain redundant information. Hence, the main challenge of this topic is how to extract useful information and reconstruct clean images. To address this problem, we propose a novel network structure, dual-encoder network with a feature fusion sub-network, to fuse feature buffers firstly, then encode the fused feature buffers and a noisy image simultaneously, and finally reconstruct a clean image by a decoder network. Compared with the state-of-the-art methods, our model is more robust on a wide range of scenes, and is able to generate satisfactory results in a significantly faster way.

Keywords

Monte Carlo renderingMonte Carlo denoisingneural network

Electronic Supplementary Material

Download File(s)
jcst-34-5-1123-Highlights.pdf (1.4 MB)
jcst-34-5-1123_ESM.pdf (1.4 MB)

References

[1]
Kajiya J T. The rendering equation. In Proc. the 13th Annual Conference on Computer Graphics and Interactive Techniques, August 1986, pp.143-150.
Crossref
[2]

Rousselle F, Manzi M, Zwicker M. Robust denoising using feature and color information. Computer Graphics Forum, 2013, 32(7): 121-130.
Crossref Google Scholar
[3]

Kalantari N K, Bako S, Sen P. A machine learning approach for filtering Monte Carlo noise. ACM Trans. Graph., 2015, 34(4): Article No. 122.
Crossref Google Scholar
[4]

Bitterli B, Rousselle F, Moon B, Guitián J A, Adler D, Mitchell K, Jarosz W, Novák J. Nonlinearly weighted first-order regression for denoising Monte Carlo renderings. Computer Graphics Forum, 2016, 35(4): 107-117.
Crossref Google Scholar
[5]

Bako S, Vogels T, McWilliams B, Meyer M, Novák J, Harvill A, Sen P, Derose T, Rousselle F. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Trans. Graph., 2017, 36(4): Article No. 97.
Crossref Google Scholar
[6]

Moon B, Carr N, Yoon S E. Adaptive rendering based on weighted local regression. ACM Transactions on Graphics, 2017, 33(5): Article No. 170.
Crossref Google Scholar
[7]
Cook R L, Porter T, Carpenter L. Distributed ray tracing. In Proc. the 11th Annual Conference on Computer Graphics and Interactive Techniques, July 1984, pp.137-145.
Crossref
[8]

Zwicker M, Jarosz W, Lehtinen J, Moon B, Ramamoorthi R, Rousselle F, Sen P, Soler C, Yoon S E. Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. Computer Graphics Forum, 2015, 34(2): 667-681.
Crossref Google Scholar
[9]

Ramamoorthi R, Mahajan D, Belhumeur P. A first-order analysis of lighting, shading, and shadows. ACM Transactions on Graphics, 2007, 26(1): Article No. 2.
Crossref Google Scholar
[10]

Jarosz W, Schönefeld V, Kobbelt L, Jensen H W. Theory, analysis and applications of 2D global illumination. ACM Transactions on Graphics, 2012, 31(5): Article No. 125.
Crossref Google Scholar
[11]

Bauszat P, Eisemann M, Magnor M. Guided image filtering for interactive high-quality global illumination. Computer Graphics Forum, 2011, 30(4): 1361-1368.
Crossref Google Scholar
[12]

Bauszat P, Eisemann M, Eisemann E, Magnor M. General and robust error estimation and reconstruction for Monte Carlo rendering. Computer Graphics Forum, 2015, 34(2): 597-608.
Crossref Google Scholar
[13]

Yang X, Wang D, Hu W, Zhao L, Piao X, Zhou D, Zhang Q, Yin B, Cai Q, Wei X. Fast reconstruction for Monte Carlo rendering using deep convolutional networks. IEEE Access, 2019, 7: 21177-21187.
Crossref Google Scholar
[14]

Chaitanya C R A, Kaplanyan A S, Schied C, Salvi M, Lefohn A, Nowrouzezahrai D, Aila T. Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. ACM Transactions on Graphics, 2017, 36(4): Article No. 98.
Crossref Google Scholar
[15]

Zhang K, Zuo W, Chen Y, Meng D, Zhang L. Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 2017, 26(7): 3142-3155.
Crossref Google Scholar
[16]

Gharbi M, Chaurasia G, Paris S, Durand F. Deep joint demosaicking and denoising. ACM Transactions on Graphics, 2016, 35(6): Article No. 191.
Crossref Google Scholar
[17]
Mao X, Shen C, Yang Y B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In Proc. the 2016 Annual Conference on Neural Information Processing Systems, December 2016, pp.2802-2810.
[18]

Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks. Science, 2016, 313(5786): 504-507.
Crossref Google Scholar
[19]
Balduzzi D, McWilliams B, Butler-Yeoman T. Neural Taylor approximations: Convergence and exploration in rectifier networks. In Proc. the 34th International Conference on Machine Learning, August 2017, pp.351-360.
[20]
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.770-778.
Crossref
[21]

Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
Crossref Google Scholar
[22]

Pharr M, Jakob W, Humphreys G. Physically Based Rendering: From Theory to Implementation (3rd edition). Morgan Kaufmann, 2016.
[23]

Eilertsen G, Kronander J, Denes G, Mantiuk R K, Unger J. HDR image reconstruction from a single exposure using deep CNNs. ACM Transactions on Graphics, 2107, 36(6): Article No. 178.
Crossref Google Scholar
[24]

Rousselle F, Knaus C, Zwicker. Adaptive sampling and reconstruction using greedy error minimization. ACM Transactions on Graphics, 2011, 30(6): Article No. 159.
Crossref Google Scholar
[25]
Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. the 13th International Conference on Artificial Intelligence and Statistics, May 2010, pp.249-256
[26]
Abadi M, Barham P, Chen J et al. Tensorflow: A system for large-scale machine learning. In Proc. the 12th USENIX Symposium on Operating Systems Design and Implementation, November 2016, pp.265-283.
[27]
Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv: 1412.6980, 2014. https://arxiv.org/abs/1412.6980, May 2019.
[28]

Boughida M, Boubekeur T. Bayesian collaborative denoising for Monte Carlo rendering. Computer Graphics Forum, 2017, 36(4): 137-153.
Crossref Google Scholar
[29]

Lang M, Wang O, Aydin T, Smolic A, Gross M. Practical temporal consistency for image-based graphics applications. ACM Transactions on Graphics, 2012, 31(4): Article No. 34.
Crossref Google Scholar
[30]

Bonneel N, Tompkin J, Sunkavalli K, Sun D, Paris S, Pfister H. Blind video temporal consistency. ACM Transactions on Graphics, 2015, 34(6): Article No. 196.
Crossref Google Scholar
[31]
Lai W S, Huang J B, Wang O, Shechtman E, Yumer E, Yang M H. Learning blind video temporal consistency. In Proc. the 15th European Conference on Computer Vision, September 2018, pp.179-185.
Crossref
Journal of Computer Science and Technology
Volume 34 Issue 5,
September 2019
Pages 1123-1135
DOI: 10.1007/s11390-019-1964-2
Cite this article:
Yang X, Wang D, Hu W, et al. DEMC: A Deep Dual-Encoder Network for Denoising Monte Carlo Rendering. Journal of Computer Science and Technology, 2019, 34(5): 1123-1135. https://doi.org/10.1007/s11390-019-1964-2

398

Views

14

Crossref

N/A

Web of Science

16

Scopus

3

CSCD

Altmetrics
Received: 15 January 2019
Revised: 28 May 2019
Published: 06 September 2019
©2019 Springer Science + Business Media, LLC & Science Press, China
Return