AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Computer Science and Technology Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Regular Paper

Predicted Robustness as QoS for Deep Neural Network Models

State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China
Department of Computer Science, University of Surrey, Guilford, GU2 7XH, U.K.
Show Author Information

Abstract

The adoption of deep neural network (DNN) model as the integral part of real-world software systems necessitates explicit consideration of their quality-of-service (QoS). It is well-known that DNN models are prone to adversarial attacks, and thus it is vitally important to be aware of how robust a model’s prediction is for a given input instance. A fragile prediction, even with high confidence, is not trustworthy in light of the possibility of adversarial attacks. We propose that DNN models should produce a robustness value as an additional QoS indicator, along with the confidence value, for each prediction they make. Existing approaches for robustness computation are based on adversarial searching, which are usually too expensive to be excised in real time. In this paper, we propose to predict, rather than to compute, the robustness measure for each input instance. Specifically, our approach inspects the output of the neurons of the target model and trains another DNN model to predict the robustness. We focus on convolutional neural network (CNN) models in the current research. Experiments show that our approach is accurate, with only 10%–34% additional errors compared with the offline heavy-weight robustness analysis. It also significantly outperforms some alternative methods. We further validate the effectiveness of the approach when it is applied to detect adversarial attacks and out-of-distribution input. Our approach demonstrates a better performance than, or at least is comparable to, the state-of-the-art techniques.

Keywords

deep neural networkpredictionquality of servicerobustness

Electronic Supplementary Material

Download File(s)
jcst-35-5-999-Highlights.pdf (316 KB)

References

[1]
Andor D, Alberti C, Weiss D, Severyn A, Presta A, Ganchev K, Petrov S, Collins M. Globally normalized transition-based neural networks. arXiv:1603.06042, 2016. https://arxiv.org/abs/1603.06042, June 2020.
Crossref
[2]

Hinton G, Deng L, Yu D, Dahl G, Mohamed A, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Kingsbury B. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 2012, 29(6): 82-97.
Crossref Google Scholar
[3]
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proc. the IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.770-778.
Crossref
[4]

Wang X, Huang C, Yao L, Benatallah B, Dong M. A survey on expert recommendation in community question answering. Journal of Computer Science and Technology, 2018, 33(4): 625-653.
Crossref Google Scholar
[5]

Liu Q, Zhao H K, Wu L, Li Z, Chen E H. Illuminating recommendation by understanding the explicit item relations. Journal of Computer Science and Technology, 2018, 33(4): 739-755.
Crossref Google Scholar
[6]

Silver D, Huang A, Maddison C J et al. Mastering the game of Go with deep neural networks and tree search. Nature, 2016, 529(7587): 484-489.
Crossref Google Scholar
[7]

Ameur H, Jamoussi S, Hamadou A B. A new method for sentiment analysis using contextual auto-encoders. Journal of Computer Science and Technology, 2018, 33(6): 1307-1319.
Crossref Google Scholar
[8]
Bojarski M, Testa D D, Dworakowski D et al. End to end learning for self-driving cars. arXiv:1604.07316, 2016. https://arxiv.org/abs/1604.07316, June 2020.
[9]

Esteva A, Kuprel B, Novoa R A, Ko J, Swetter S M, Blau H M, Thrun S. Dermatologist-level classification of skin cancer with deep neural networks. Nature, 2017, 542(7639): 115-118.
Crossref Google Scholar
[10]

Yuan Z, Lu Y, Wang Z, Xue Y. Droid-Sec: Deep learning in Android malware detection. ACM SIGCOMM Computer Communication Review, 2014, 44(4): 371-372.
Crossref Google Scholar
[11]
Li Z, Ma X, Xu C, Xu J, Cao C, Lü J. Operational calibration: Debugging confidence errors for DNNs in the field. arXiv:1910.02352, 2019. https://arxiv.org/abs/1910.02352, Sept. 2020.
Crossref
[12]
Li Z, Ma X, Xu C, Cao C, Xu J, Lü J. Boosting operational DNN testing efficiency through conditioning. In Proc. the 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, August 2019, pp.499-509.
Crossref
[13]
LeCun Y, Bengio Y, Hinton G. Deep Learning. MIT Press, 2016.
Crossref
[14]

Burrell J. How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society, 2016, 3(1): Article No. 2053951715622512.
Crossref Google Scholar
[15]
Goodfellow I J, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. arXiv:1412.6572, 2014. https://arxiv.org/abs/1412.6572, June 2020.
[16]
Moosavi-Dezfooli S, Fawzi A, Frossard P. DeepFool: A simple and accurate method to fool deep neural networks. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.2574-2582.
Crossref
[17]
Carlini N, Wagner D. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proc. the 10th ACM Workshop on Artificial Intelligence and Security, November 2017, pp.3-14.
Crossref
[18]
Athalye A, Carlini N, Wagner D. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. arXiv:1802.00420, 2018. https://arxiv.org/abs/1802.00420, June 2020.
[19]
Katz G, Barrett C, Dill D L, Julian K, Kochenderfer M J. Reluplex: An efficient SMT solver for verifying deep neural networks. In Proc. the 29th International Conference on Computer Aided Verification, July 2017, pp.97-117.
Crossref
[20]
Bastani O, Ioannou Y, Lampropoulos L, Vytiniotis D, Nori A, Criminisi A. Measuring neural net robustness with constraints. In Proc. the Annual Conference on Neural Information Processing Systems, December 2016, pp.2613-2621.
[21]
Hendrycks D, Gimpel K. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv:1610.02136, 2016. https://arxiv.org/abs/1610.02136, June 2020.
[22]
Weng T, Zhang H, Chen H, Song Z, Hsieh C, Boning D, Dhillon I S, Daniel L. Towards fast computation of certified robustness for ReLU networks. arXiv:1804.09699, 2018. https://arxiv.org/abs/1804.09699, June 2020.
[23]

Singh G, Gehr T, Püschel M, Vechev M. An abstract domain for certifying neural networks. Proceedings of the ACM on Programming Languages, 2019, 3(POPL): Article No. 41.
Crossref Google Scholar
[24]
Carlini N, Wagner D. Towards evaluating the robustness of neural networks. In Proc. the 2017 IEEE Symposium on Security and Privacy, May 2017, pp.39-57.
Crossref
[25]
Feinman R, Curtin R R, Shintre S, Gardner A B. Detecting adversarial samples from artifacts. arXiv:1703.00410, 2017. https://arxiv.org/abs/1703.00410, June 2020.
[26]
Ma X, Li B, Wang Y, Erfani S M, Wijewickrema S, Schoenebeck G, Song D, Houle M E, Bailey J. Characterizing adversarial subspaces using local intrinsic dimensionality. arXiv:1801.02613, 2018. https://arxiv.org/abs/1801.02613, June 2020.
[27]
Wang Y, Li Z, Xu J, Yu P, Ma X. Fast robustness prediction for deep neural network. In Proc. the 11th Asia-Pacific Symposium on Internetware, Oct. 2019.
Crossref
[28]
Kurakin A, Goodfellow I, Bengio S. Adversarial examples in the physical world. arXiv:1607.02533, 2016. https://arxiv.org/abs/1607.02533, June 2020.
[29]
Papernot N, McDaniel P, Jha S, Fredrikson M, Celik Z B, Swami A. The limitations of deep learning in adversarial settings. In Proc. the 2016 IEEE European Symposium on Security and Privacy, March 2016, pp.372-387.
Crossref
[30]
Huang X, Kroening D, Kwiatkowska M, Ruan W, Sun Y, Thamo E, Wu M, Yi X. Safety and trustworthiness of deep neural networks: A survey. arXiv:1812.08342, 2018. https://arxiv.org/abs/1812.08342, June 2020.
[31]
Huang X, Kwiatkowska M, Wang S, Wu M. Safety verification of deep neural networks. In Proc. the 29th International Conference on Computer Aided Verification, July 2017, pp.3-29.
Crossref
[32]
Wong E, Kolter J Z. Provable defenses against adversarial examples via the convex outer adversarial polytope. arXiv:1711.00851, 2017. https://arxiv.org/abs/1711.00851, June 2020.
[33]
Gopinath D, Pasareanu C S, Wang K, Zhang M, Khurshid S. Symbolic execution for attribution and attack synthesis in neural networks. In Proc. the 41st IEEE/ACM International Conference on Software Engineering, May 2019, pp.282-283.
Crossref
[34]
Pei K, Cao Y, Yang J, Jana S. DeepXplore: Automated whitebox testing of deep learning systems. In Proc. the 26th Symposium on Operating Systems Principles, October 2017, pp.1-18.
Crossref
[35]
Ma L, Juefei-Xu F, Zhang F et al. DeepGauge: Multigranularity testing criteria for deep learning systems. In Proc. the 33rd ACM/IEEE International Conference on Automated Software Engineering, September 2018, pp.120-131.
Crossref
[36]
Ma L, Zhang F, Xue M, Li B, Liu Y, Zhao J, Wang Y. Combinatorial testing for deep learning systems. arXiv:1806.07723, 2018. https://arxiv.org/abs/1806.07723, June 2020.
[37]
Zong B, Song Q, Min M, Cheng W, Lumezanu C, Cho D, Chen H. Deep autoencoding Gaussian mixture model for unsupervised anomaly detection. In Proc. International Conference on Learning Representations, February 2018.
[38]
Santhanam G K, Grnarova P. Defending against adversarial attacks by leveraging an entire GAN. arXiv:1805.10652, 2018. https://arxiv.org/abs/1805.10652, June 2020.
[39]
Grosse K, Manoharan P, Papernot N, Backes M, McDaniel P. On the (statistical) detection of adversarial examples. arXiv:1702.06280, 2017. https://arxiv.org/abs/1702.06280, June 2020.
[40]
Xu W, Evans D, Qi Y. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv:1704.01155, 2017. https://arxiv.org/abs/1704.01155, June 2020.
Crossref
[41]
Benesty J, Chen J, Huang Y, Cohen I. Pearson correlation coefficient. In Noise Reduction in Speech Processing, Cohen I, Huang Y, Chen J, Benesty J (eds.), Springer, 2009, pp.1-4.
Crossref
[42]

LeCun L, Boser B, Denker J S, Henderson D, Howard R E, Hubbard W, Jackel L D. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1989, 1(4): 541-551.
Crossref Google Scholar
[43]
Krizhevsky A. Learning multiple layers of features from tiny images. Technical Report, University of Toronto, 2009. http://www.cs.toronto.edu/_kriz/learning-features-2009-TR.pdf, June 2020.
[44]
Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng A Y. Reading digits in natural images with unsupervised feature learning. In Proc. the NIPS Workshop on Deep Learning and Unsupervised Feature Learning, Dec. 2011.
[45]
Deng J, Dong W, Socher R, Li L J, Li K, Li F F. ImageNet: A large-scale hierarchical image database. In Proc. the 2009 IEEE Conference on Computer Vision and Pattern Recognition, June 2009, pp.248-255.
Crossref
[46]

LeCun L, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
Crossref Google Scholar
[47]
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556, 2014. https://arxiv.org/abs/1409.1556, June 2020.
[48]
Kim J, Feldt R, Yoo S. Guiding deep learning system testing using surprise adequacy. In Proc. the 41st International Conference on Software Engineering, May 2019, pp.1039-1049.
Crossref
Journal of Computer Science and Technology
Volume 35 Issue 5,
September 2020
Pages 999-1015
DOI: 10.1007/s11390-020-0482-6
Cite this article:
Wang Y-H, Li Z-N, Xu J-W, et al. Predicted Robustness as QoS for Deep Neural Network Models. Journal of Computer Science and Technology, 2020, 35(5): 999-1015. https://doi.org/10.1007/s11390-020-0482-6

453

Views

4

Crossref

N/A

Web of Science

5

Scopus

0

CSCD

Altmetrics
Received: 31 March 2020
Revised: 29 July 2020
Published: 30 September 2020
©Institute of Computing Technology, Chinese Academy of Sciences 2020
Return