AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (1 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Univariate Time Series Anomaly Detection Based on Hierarchical Attention Network

State Grid Beijing Electric Power Company, Beijing 100031, China
Institute of Electrical Engineering, Chinese Academy of Sciences, Beijing 100190, China
Show Author Information

Abstract

In order to support the perception and defense of the operation risk of the medium and low voltage distribution system, it is crucial to conduct data mining on the time series generated by the system to learn anomalous patterns, and carry out accurate and timely anomaly detection for timely discovery of anomalous conditions and early alerting. And edge computing has been widely used in the processing of Internet of Things (IoT) data. The key challenge of univariate time series anomaly detection is how to model complex nonlinear time dependence. However, most of the previous works only model the short-term time dependence, without considering the periodic long-term time dependence. Therefore, we propose a new Hierarchical Attention Network (HAN), which introduces seven day-level attention networks to capture fine-grained short-term time dependence, and uses a week-level attention network to model the periodic long-term time dependence. Then we combine the day-level feature learned by day-level attention network and week-level feature learned by week-level attention network to obtain the high-level time feature, according to which we can calculate the anomaly probability and further detect the anomaly. Extensive experiments on a public anomaly detection dataset, and deployment in a real-world medium and low voltage distribution system show the superiority of our proposed framework over state-of-the-arts.

Keywords

anomaly detectionunivariate time seriesself-attentionedge computing

References

[1]

R. A. Leon, V. Vittal, and G. Manimaran, Application of sensor network for secure electric energy infrastructure, IEEE Trans. Power Deliv., vol. 22, no. 2, pp. 1021–1028, 2007.
Crossref Google Scholar
[2]
N. Laptev, S. Amizadeh, and I. Flint, Generic and scalable framework for automated time-series anomaly detection, in Proc. 21th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Sydney, Australia, 2015, pp. 1939–1947.
Crossref
[3]
Z. Li, X. Xu, T. Hang, H. Xiang, Y. Cui, L. Qi, and X. Zhou, A knowledge-driven anomaly detection framework for social production system, IEEE Trans. Comput. Soc. Syst. doi: 10.1109/TCSS.2022.3217790.
Crossref
[4]

Y. Jia, B. Liu, W. Dou, X. Xu, X. Zhou, L. Qi, and Z. Yan, CroApp: A CNN-based resource optimization approach in edge computing environment, IEEE Trans. Ind. Inform., vol. 18, no. 9, pp. 6300–6307, 2022.
Crossref Google Scholar
[5]

S. Chen, Y. Tao, D. Yu, F. Li, and B. Gong, Distributed learning dynamics of multi-armed bandits for edge intelligence, J. Syst. Archit., vol. 114, p. 101919, 2021.
Crossref Google Scholar
[6]

S. Chen, Y. Tao, D. Yu, F. Li, B. Gong, and X. Cheng, Privacy-preserving collaborative learning for multiarmed bandits in IoT, IEEE Internet Things J., vol. 8, no. 5, pp. 3276–3286, 2021.
Crossref Google Scholar
[7]

H. Yu, Z. Chen, X. Zhang, X. Chen, F. Zhuang, H. Xiong, and X. Cheng, FedHAR: Semi-supervised online learning for personalized federated human activity recognition, IEEE Trans. Mob. Comput., vol. 22, no. 6, pp. 3318–3332, 2023.
Crossref Google Scholar
[8]

V. Zeufack, D. Kim, D. Seo, and A. Lee, An unsupervised anomaly detection framework for detecting anomalies in real time through network system’s log files analysis, High Confid. Comput., vol. 1, no. 2, p. 100030, 2021.
Crossref Google Scholar
[9]

J. Chen, H. Huang, and H. Chen, Informer: Irregular traffic detection for containerized microservices RPC in the real world, High Confid. Comput., vol. 2, no. 2, p. 100050, 2022.
Crossref Google Scholar
[10]

S. Wu, S. Shen, X. Xu, Y. Chen, X. Zhou, D. Liu, X. Xue, and L. Qi, Popularity-aware and diverse web APIs recommendation based on correlation graph, IEEE Trans. Comput. Soc. Syst., vol. 10, no. 2, pp. 771–782, 2023.
Crossref Google Scholar
[11]

X. Zhou, X. Xu, W. Liang, Z. Zeng, and Z. Yan, Deep-learning-enhanced multitarget detection for end-edge-cloud surveillance in smart IoT, IEEE Internet Things J., vol. 8, no. 16, pp. 12588–12596, 2021.
Crossref Google Scholar
[12]

L. Qi, Y. Yang, X. Zhou, W. Rafique, and J. Ma, Fast anomaly identification based on multiaspect data streams for intelligent intrusion detection toward secure industry 4.0, IEEE Trans. Ind. Inform., vol. 18, no. 9, pp. 6503–6511, 2022.
Crossref Google Scholar
[13]
H. Z. Moayedi and M. A. Masnadi-Shirazi, ARIMA model for network traffic prediction and anomaly detection, in Proc. 2008 Int. Symp. Information Technology, Kuala Lumpur, Malaysia, 2008, pp. 1–6.
Crossref
[14]
F. Knorn and D. J. Leith, Adaptive Kalman filtering for anomaly detection in software appliances, in Proc. IEEE INFOCOM Workshops, Phoenix, AZ, USA, 2008, pp. 1–6.
Crossref
[15]
M. Amer, M. Goldstein, and S. Abdennadher, Enhancing one-class support vector machines for unsupervised anomaly detection, in Proc. ACM SIGKDD Workshop on Outlier Detection and Description, Chicago, IL, USA, 2013, pp. 8–15.
Crossref
[16]

S. M. Erfani, S. Rajasegarar, S. Karunasekera, and C. Leckie, High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning, Pattern Recognit., vol. 58, pp. 121–134, 2016.
Crossref Google Scholar
[17]

Z. Ding and M. Fei, An anomaly detection approach based on isolation forest algorithm for streaming data using sliding window, IFAC Proc. Vol., vol. 46, no. 20, pp. 12–17, 2013.
Crossref Google Scholar
[18]
F. Angiulli and C. Pizzuti, Fast outlier detection in high dimensional spaces, in Proc. 6th European Conf. Principles of Data Mining and Knowledge Discovery, 2002, Helsinki, Finland, 2002, pp. 15–27.
Crossref
[19]
R. Zhang, S. Dong, X. Nie, and S. Xiao, Feedforward neural network for time series anomaly detection, arXiv preprint arXiv: 1812.08389, 2018.
[20]

S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Comput., vol. 9, no. 8, pp. 1735–1780, 1997.
Crossref Google Scholar
[21]

H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, Informer: Beyond efficient transformer for long sequence time-series forecasting, Proc. AAAI Conf. Artif. Intell., vol. 35, no. 12, pp. 11106–11115, 2021.
Crossref Google Scholar
[22]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need, in Proc. 31st Int. Conf. on Neural Information Processing Systems, Long Beach, CA, USA, 2017, pp. 6000–6010.
[23]

Z. Chen, D. Chen, X. Zhang, Z. Yuan, and X. Cheng, Learning graph structures with transformer for multivariate time-series anomaly detection in IoT, IEEE Internet Things J., vol. 9, no. 12, pp. 9179–9189, 2022.
Crossref Google Scholar
[24]
P. Malhotra, L. Vig, G. M. Shroff, and P. Agarwal, Long short term memory networks for anomaly detection in time series, in Proc. 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium, 2015, pp. 89–94.
[25]

M. Munir, S. A. Siddiqui, A. Dengel, and S. Ahmed, DeepAnT: A deep learning approach for unsupervised anomaly detection in time series, IEEE Access, vol. 7, pp. 1991–2005, 2018.
Crossref Google Scholar
[26]
C. C. Aggarwal, Time series and multidimensional streaming outlier detection, in Outlier Analysis, C. C. Aggarwal, ed. Cham, Switzerland: Springer, 2017, pp. 273–310.
Crossref
[27]
H. Xu, W. Chen, N. Zhao, Z. Li, J. Bu, Z. Li, Y. Liu, Y. Zhao, D. Pei, Y. Feng, et al., Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications, in Proc. 2018 World Wide Web Conf., Lyon, France, 2018, pp. 187–196.
Crossref
[28]

D. Park, Y. Hoshi, and C. C. Kemp, A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder, IEEE Robot. Autom. Lett., vol. 3, no. 3, pp. 1544–1551, 2018.
Crossref Google Scholar
[29]
D. Li, D. Chen, B. Jin, L. Shi, J. Goh, and S. K. Ng, MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks, in Proc. 28th Int. Conf. Artificial Neural Networks, Munich, Germany, 2019, pp. 703–716.
Crossref
[30]
J. Audibert, P. Michiardi, F. Guyard, S. Marti, and M. A. Zuluaga, USAD: Unsupervised anomaly detection on multivariate time series, in Proc. 26th ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining, Virtual Event, CA, USA, 2020, pp. 3395–3404.
[31]

Y. Dwivedi and S. S. Rao, A test for second-order stationarity of a time series based on the discrete Fourier transform, J. Time Ser. Anal., vol. 32, no. 1, pp. 68–91, 2011.
Crossref Google Scholar
[32]

L. Breiman, Random forests, Mach. Learn., vol. 45, no. 1, pp. 5–32, 2001.
Crossref Google Scholar
[33]

M. A. Hearst, S. T. Dumais, E. Osuna, J. Platt, and B. Scholkopf, Support vector machines, IEEE Intell. Syst. Appl., vol. 13, no. 4, pp. 18–28, 1998.
Crossref Google Scholar
[34]
D. Liu, Y. Zhao, H. Xu, Y. Sun, D. Pei, J. Luo, X. Jing, and M. Feng, Opprentice: Towards practical and automatic anomaly detection through machine learning, in Proc. 2015 Internet Measurement Conference, Tokyo, Japan, 2015, pp. 211–224.
Crossref
[35]
W. Qiu, Y. Wu, G. Wang, S. X. Yang, J. Bai, and J. Li, A novel unsupervised anomaly detection based on robust principal component classifier, in Proc. Defense and Security Symposium, Orlando (Kissimmee), FL, USA, 2006, pp. 260–268.
Crossref
[36]

S. Chen, Y. Wang, D. Yu, J. Ren, C. Xu, and Y. Zheng, Privacy-enhanced decentralized federated learning at dynamic edge, IEEE Trans. Comput., vol. 72, no. 8, pp. 2165–2180, 2023.
Crossref Google Scholar
[37]
Y. Yuan, J. Yu, X. Cheng, Z. Zou, D. Yu, and Z. Cai, Decentralized parallel SGD based on weight-balancing for intelligent IoV, IEEE Trans. Intell. Transp. Syst. doi: 10.1109/TITS.2022.3216709.
Crossref
[38]

Z. Xie, Y. Huang, D. Yu, R. M. Parizi, Y. Zheng, and J. Pang, FedEE: A federated graph learning solution for extended enterprise collaboration, IEEE Trans. Ind. Inform., vol. 19, no. 7, pp. 8061–8071, 2023.
Crossref Google Scholar
[39]

S. Chen, D. Yu, Y. Zou, J. Yu, and X. Cheng, Decentralized wireless federated learning with differential privacy, IEEE Trans. Ind. Inform., vol. 18, no. 9, pp. 6273–6282, 2022.
Crossref Google Scholar
[40]
S. Chen, D. Yu, F. Li, Z. Zou, W. Liang, and X. Cheng, PPAR: A privacy-preserving adaptive ranking algorithm for multi-armed-bandit crowdsourcing, in Proc. 2022 IEEE/ACM 30th Int. Symp. Quality of Service (IWQoS), Oslo, Norway, 2022, pp. 1–10.
Crossref
[41]

D. Yu, Z. Zou, S. Chen, Y. Tao, B. Tian, W. Lv, and X. Cheng, Decentralized parallel SGD with privacy preservation in vehicular networks, IEEE Trans. Veh. Technol., vol. 70, no. 6, pp. 5211–5220, 2021.
Crossref Google Scholar
[42]

Y. Liu, D. Li, S. Wan, F. Wang, W. Dou, X. Xu, S. Li, R. Ma, and L. Qi, A long short-term memory-based model for greenhouse climate prediction, Int. J. Intell. Syst., vol. 37, no. 1, pp. 135–151, 2022.
Crossref Google Scholar
[43]

Y. Liu, H. Wu, K. Rezaee, M. R. Khosravi, O. I. Khalaf, A. A. Khan, D. Ramesh, and L. Qi, Interaction-enhanced and time-aware graph convolutional network for successive point-of-interest recommendation in traveling enterprises, IEEE Trans. Ind. Inform., vol. 19, no. 1, pp. 635–643, 2023.
Crossref Google Scholar
[44]

L. Qi, Y. Liu, Y. Zhang, X. Xu, M. Bilal, and H. Song, Privacy-aware point-of-interest category recommendation in Internet of Things, IEEE Internet Things J., vol. 9, no. 21, pp. 21398–21408, 2022.
Crossref Google Scholar
[45]
A. Ihler, J. Hutchins, and P. Smyth, Adaptive event detection with time-varying Poisson processes, in Proc. 12th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Philadelphia, PA, USA, 2006, pp. 207–216.
Crossref
Tsinghua Science and Technology
Volume 29 Issue 4,
August 2024
Pages 1181-1193
DOI: 10.26599/TST.2023.9010073
Cite this article:
Chen Z, Jia D, Sun Y, et al. Univariate Time Series Anomaly Detection Based on Hierarchical Attention Network. Tsinghua Science and Technology, 2024, 29(4): 1181-1193. https://doi.org/10.26599/TST.2023.9010073

1167

Views

252

Downloads

1

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 01 June 2023
Revised: 19 June 2023
Accepted: 13 July 2023
Published: 09 February 2024
© The Author(s) 2024.

The articles published in this open access journal are distributed under the terms of theCreative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return