AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Big Data Mining and Analytics Article
PDF (647.4 KB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Attention-Aware Heterogeneous Graph Neural Network

College of Sciences, Northeastern University, Shenyang 110004, China
State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University, Shenyang 110819, China
Show Author Information

Abstract

As a powerful tool for elucidating the embedding representation of graph-structured data, Graph Neural Networks (GNNs), which are a series of powerful tools built on homogeneous networks, have been widely used in various data mining tasks. It is a huge challenge to apply a GNN to an embedding Heterogeneous Information Network (HIN). The main reason for this challenge is that HINs contain many different types of nodes and different types of relationships between nodes. HIN contains rich semantic and structural information, which requires a specially designed graph neural network. However, the existing HIN-based graph neural network models rarely consider the interactive information hidden between the meta-paths of HIN in the poor embedding of nodes in the HIN. In this paper, we propose an Attention-aware Heterogeneous graph Neural Network (AHNN) model to effectively extract useful information from HIN and use it to learn the embedding representation of nodes. Specifically, we first use node-level attention to aggregate and update the embedding representation of nodes, and then concatenate the embedding representation of the nodes on different meta-paths. Finally, the semantic-level neural network is proposed to extract the feature interaction relationships on different meta-paths and learn the final embedding of nodes. Experimental results on three widely used datasets showed that the AHNN model could significantly outperform the state-of-the-art models.

Keywords

Graph Neural Network (GNN)Heterogeneous Information Network (HIN)embedding

References

[1]
D. Duvenaud, D. Maclaurin, J. Aguilera-Iparraguirre, R. Gómez-Bombarelli, T. Hirzel, A. Aspuru-Guzik, and R. P. Adams, Convolutional networks on graphs for learning molecular fingerprints, in Proc. 28th Int. Conf. Neural Information Processing Systems, Montreal, Canada, 2015, pp. 2224-2232.
[2]
J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, Neural message passing for quantum chemistry, in Proc. 34th Int. Conf. Machine Learning, Sydney, Australia, 2017, pp. 1263-1277.
[3]
M. D. Cranmer, R. Xu, P. Battaglia, and S. Ho, Learning symbolic physics with graph networks, arXiv preprint arXiv: 1909.05862, 2019.
Google Scholar
[4]
A. Sanchez-Gonzalez, N. Heess, J. T. Springenberg, J. Merel, M. Riedmiller, R. Hadsell, and P. Battaglia, Graph networks as learnable physics engines for inference and control, in Proc. 35th Int. Conf. Machine Learning, Stockholm, Sweden, 2018, pp. 4470-4479.
[5]
T. N. Kipf and M. Welling, Semi-supervised classification with graph convolutional networks, in Proc. 5th Int. Conf. Learning Representations, Toulon, France, 2017, https://openreview.net/forum?id=SJU4ayYgl.
[6]
X. N. He, K. Deng, X. Wang, Y. Li, Y. D. Zhang, and M. Wang, LightGCN: Simplifying and powering graph convolution network for recommendation, in Proc. 43rdInt. ACM SIGIR Conf. Research and Development in Information Retrieval, .
Crossref
[7]
V. P. Dwivedi, C. K. Joshi, T. Laurent, Y. Bengio, and X. Breson, Benchmarking graph neural networks, arXiv preprint arXiv: 2003.00982, 2020.
Google Scholar
[8]
Z. W. Zhang, P. Cui, and W. W. Zhu, Deep learning on graphs: A survey, IEEE Transactions on Knowledge and Data Engineering, .
Crossref Google Scholar
[9]
P. Velickovie, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, Graph attention networks, in Proc. 6th Int. Conf. Learning Representations, Vancouver, Canada, 2018, https://openreview.net/forum?id=rJXMpikCZ.
[10]
K. Xu, W. H. Hu, J. Leskovec, and S. Jegelka, How powerful are graph neural networks? in Proc. 7th Int. Conf. Learning Representations, New Orleans, LA, USA, 2019, https://openreview.net/forum?id=ryGs6iA5Km.
[11]
H. M. Zhu, F. L. Feng, X. N. He, X. Wang, Y. Li, K. Zheng, and Y. D. Zhang, Bilinear graph neural network with neighbor interactions, in Proc. 29th Int. Joint Conf. Artificial Intelligence, Yokohama, Japan, 2020, pp. 1452-1458.
Crossref
[12]
C. Shi, B. B. Hu, W. X. Zhao, and P. S. Yu, Heterogeneous information network embedding for recommendation, IEEE Transactions on Knowledge and Data Engineering, vol. 31, no. 2, pp. 357-370, 2019.
Crossref Google Scholar
[13]
X. Wang, M. Q. Zhu, D. Y. Bo, P. Cui, C. Shi, and J. Pei, AM-GCN: Adaptive multi-channel graph convolutional networks, in Proc. 26th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, San Diego, CA, USA, 2020, pp. 1243-1253.
Crossref
[14]
X. Wang, H. Y. Ji, C. Shi, B. Wang, Y. F. Ye, P. Cui, and P. S. Yu, Heterogeneous graph attention network, in Proc. of the World Wide Web Conf., San Francisco, CA, USA, 2019, pp. 2022-2032.
Crossref
[15]
W. J. Chen, Y. L. Gu, Z. C. Ren, X. N. He, H. T. Xie, T. Guo, D. W. Yin, and Y. D. Zhang, Semi-supervised user profiling with heterogeneous graph attention networks, in Proc. 28th Int. Joint Conf. Artificial Intelligence, Macao, China, 2019, pp. 2116-2122.
Crossref
[16]
H. T. Hong, H. T. Guo, Y. C. Lin, X. Q. Yang, Z. Li, and J. P. Ye, An attention-based graph neural network for heterogeneous structural learning, in Proc. 34th AAAI Conf. Artificial Intelligence, New York, NY, USA, 2020, pp. 4132-4139.
Crossref
[17]
S. Zhang and L. Xie, Improving attention mechanism in graph neural networks via cardinality preservation, in Proc. 29th Int. Joint Conf. Artificial Intelligence, New York, NY, USA, 2020, pp. 1395-1402.
Crossref
[18]
B. Perozzi, R. Al-Rfou, and S. Skiena, DeepWalk: Online learning of social representations, in Proc. 20th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, New York, NY, USA, 2014, pp. 701-710.
Crossref
[19]
J. Tang, M. Qu, M. Z. Wang, M. Zhang, J. Yan, and Q. Z. Mei, LINE: Large-scale information network embedding, in Proc. 24th Int. Conf. World Wide Web, Florence, Italy, 2015, pp. 1067-1077.
Crossref
[20]
Y. X. Dong, N. V. Chawla, and A. Swami, Metapath2vec: Scalable representation learning for heterogeneous networks, in Proc. 23rd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Halifax, Canada, 2017, pp. 135-144.
Crossref
[21]
Z. N. Hu, Y. X. Dong, K. S. Wang, and Y. Z. Sun, Heterogeneous graph transformer, in Proc. Web Conf., Taipei, China, 2020, pp. 2704-2710.
Crossref
[22]
X. Y. Fu, J. N. Zhang, Z. Q. Meng, and I. King, MAGNN: Metapath aggregated graph neural network for heterogeneous graph embedding, in Proc. Web Conf., Taipei, China, 2020, pp. 2331-2341.
Crossref
Big Data Mining and Analytics
Volume 4 Issue 4,
December 2021
Pages 233-241
DOI: 10.26599/BDMA.2021.9020008
Cite this article:
Zhang J, Xu Q. Attention-Aware Heterogeneous Graph Neural Network. Big Data Mining and Analytics, 2021, 4(4): 233-241. https://doi.org/10.26599/BDMA.2021.9020008

1665

Views

1085

Downloads

28

Crossref

19

Web of Science

27

Scopus

1

CSCD

Altmetrics
Received: 21 February 2021
Revised: 25 April 2021
Accepted: 28 April 2021
Published: 26 August 2021
© The author(s) 2021

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return