AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (6.1 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Restage: Relation Structure-Aware Hierarchical Heterogeneous Graph Embedding

School of Computer Science and Technology, Anhui University, and also with Information Materials and Intelligent Sensing Laboratory of Anhui Province, Hefei 230601, China

Show Author Information

Abstract

Heterogeneous graphs contain multiple types of entities and relations, which are capable of modeling complex interactions. Embedding on heterogeneous graphs has become an essential tool for analyzing and understanding such graphs. Although these meticulously designed methods make progress, they are limited by model design and computational resources, making it difficult to scale to large-scale heterogeneous graph data and hindering the application and promotion of these methods. In this paper, we propose Restage, a relation structure-aware hierarchical heterogeneous graph embedding framework. Under this framework, embedding only a smaller-scale graph with existing graph representation learning methods is sufficient to obtain node representations on the original heterogeneous graph. We consider two types of relation structures in heterogeneous graphs: interaction relations and affiliation relations. Firstly, we design a relation structure-aware coarsening method to successively coarsen the original graph to the top-level layer, resulting in a smaller-scale graph. Secondly, we allow any unsupervised representation learning methods to obtain node embeddings on the top-level graph. Finally, we design a relation structure-aware refinement method to successively refine the node embeddings from the top-level graph back to the original graph, obtaining node embeddings on the original graph. Experimental results on three public heterogeneous graph datasets demonstrate the enhanced scalability of representation learning methods by the proposed Restage. On another large-scale graph, the speed of existing representation learning methods is increased by up to eighteen times at most.

Keywords

heterogeneous graphgraph embeddingrelation structurehierarchical

References

[1]

C. Yang, Y. Xiao, Y. Zhang, Y. Sun, and J. Han, Heterogeneous network representation learning: A unified framework with survey and benchmark, IEEE Trans. Knowl. Data Eng., vol. 34, no. 10, pp. 4854–4873, 2022.
Crossref Google Scholar
[2]

X. Wang, D. Bo, C. Shi, S. Fan, Y. Ye, and P. S. Yu, A survey on heterogeneous graph embedding: Methods, techniques, applications and sources, IEEE Trans. Big Data, vol. 9, no. 2, pp. 415–436, 2023.
Crossref Google Scholar
[3]
B. Perozzi, R. Al-Rfou, and S. Skiena, DeepWalk: Online learning of social representations, in Proc. 20th ACM SIGKDD Int. Conf. Knowledge discovery and data mining, New York, NY, USA, 2014, pp. 701–710.
Crossref
[4]
X. Wang, X. He, Y. Cao, M. Liu, and T. S. Chua, KGAT: Knowledge graph attention network for recommendation, in Proc. 25th ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining, Anchorage, AK, USA, 2019, pp. 950–958.
Crossref
[5]

A. Xu, P. Zhong, Y. Kang, J. Duan, A. Wang, M. Lu, and C. Shi, THAN: Multimodal transportation recommendation with heterogeneous graph attention networks, IEEE Trans. Intell. Transport. Syst., pp. 1–11, 2022.
Crossref Google Scholar
[6]

J. Liang, S. Gurukar, and S. Parthasarathy, MILE: A multi-level framework for scalable graph embedding, Proc. Int. AAAI Conf. Web Soc. Medium., vol. 15, pp. 361–372, 2021.
Crossref Google Scholar
[7]
C. Deng, Z. Zhao, Y. Wang, Z. Zhang, and Z. Feng, GraphZoom: A multi-level spectral approach for accurate and scalable graph embedding, in Proc. 8th Int. Conf. on Learning Representations, Addis Ababa, Ethiopia, 2020.
[8]
S. Zhao, Z. Du, J. Chen, Y. Zhang, J. Tang, and P. S. Yu, Hierarchical representation learning for attributed networks, in Proc. IEEE 38th Int. Conf. Data Engineering (ICDE ), Kuala Lumpur, Malaysia, 2022, pp. 2641–2656.
Crossref
[9]

Y. Lu, C. Shi, L. Hu, and Z. Liu, Relation structure-aware heterogeneous information network embedding, Proc. AAAI Conf. Artif. Intell., vol. 33, no. 1, pp. 4456–4463, 2019.
Crossref Google Scholar
[10]

C. Shi, Y. Lu, L. Hu, Z. Liu, and H. Ma, RHINE: Relation structure-aware heterogeneous information network embedding, IEEE Trans. Knowl. Data Eng., vol. 34, no. 1, pp. 433–447, 2022.
Crossref Google Scholar
[11]
S. Zhu, C. Zhou, S. Pan, X. Zhu, and B. Wang, Relation structure-aware heterogeneous graph neural network, in Proc. IEEE Int. Conf. Data Mining (ICDM ), Beijing, China, 2019, pp. 1534–1539.
Crossref
[12]
J. Tang, M. Qu, and Q. Mei, PTE: Predictive text embedding through large-scale heterogeneous text networks, in Proc. 21th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Sydney, Australia, 2015, pp. 1165–1174.
Crossref
[13]
Y. Dong, N. V. Chawla, and A. Swami, metapath2vec: Scalable representation learning for heterogeneous networks, in Proc. 23rd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Halifax, Canada, 2017, pp. 135–144.
Crossref
[14]
T.-Y. Fu, W.-C. Lee, and Z. Lei, HIN2Vec: Explore meta-paths in heterogeneous information networks for representation learning, in Proc. 2017 ACM on Conf. Information and Knowledge Management, Singapore, Singapore, 2017, pp. 1797–1806.
[15]
M. Defferrard, X. Bresson, and P. Vandergheynst, Convolutional neural networks on graphs with fast localized spectral filtering, in Proc. 30th Int. Conf. Neural Information Processing Systems, Barcelona, Spain, 2016, pp. 3844–3852.
[16]
X. Wang, H. Ji, C. Shi, B. Wang, Y. Ye, P. Cui, and P. S. Yu, Heterogeneous graph attention network, in Proc. World Wide Web Conference, San Francisco, CA, USA, 2019, pp. 2022–2032.
Crossref
[17]

X. Wang, Y. Zhang, and C. Shi, Hyperbolic heterogeneous information network embedding, Proc. AAAI Conf. Artif. Intell., vol. 33, no. 1, pp. 5337–5344, 2019.
Crossref Google Scholar
[18]

C. Shi, B. Hu, W. X. Zhao, and P. S. Yu, Heterogeneous information network embedding for recommendation, IEEE Trans. Knowl. Data Eng., vol. 31, no. 2, pp. 357–370, 2019.
Crossref Google Scholar
[19]
T. N. Kipf and M.Welling, Semi-supervised classification with graph convolutional networks, in Proc. 6th Int. Conf. on Learning Representations, Toulon, France, 2017.
[20]
P. Veličkovit’c, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, Graph attention networks, in Proc. 6th Int. Conf. on Learning Representations, Vancouver, Canada, 2017.
[21]
W. L. Hamilton, R. Ying, and J. Leskovec, Inductive representation learning on large graphs, in Proc. 31st Int. Conf. Neural Information Processing Systems, Long Beach, CA, USA, 2017, pp. 1025–1035.
[22]
M. Schlichtkrull, T. N. Kipf, P. Bloem, R. van den Berg, I. Titov, and M. Welling, Modeling relational data with graph convolutional networks, in Lecture Notes in Computer Science, R. D. Deshpande and R. D. Deshpande, eds. Cham, Switzerland: Springer, 2018, pp. 593–607.
Crossref
[23]
Y. Cen, X. Zou, J. Zhang, H. Yang, J. Zhou, and J. Tang, Representation learning for attributed multiplex heterogeneous network, in Proc. 25th ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining, Anchorage, AK, USA, 2019, pp. 1358–1368.
Crossref
[24]
X. Fu, J. Zhang, Z. Meng, and I. King, MAGNN: Metapath aggregated graph neural network for heterogeneous graph embedding, in Proc. The Web Conf. 2020, Taipei, China, 2020, pp. 2331–2341.
Crossref
[25]
J. Zhao, X. Wang, C. Shi, Z. Liu, and Y. Ye, Network schema preserved heterogeneous information network embedding, in Proc. 29th Int. Joint Conf. on Artificial Intelligence, Yokohama, Japan, 2020, pp. 1366–1372.
Crossref
[26]
F. Wu, A. Souza, T. Zhang, C. Fifty, T. Yu, and K. Weinberger, Simplifying graph convolutional networks, in Proc. 36th Int. Conf. on Machine Learning, Long Beach, CA, USA, 2019, pp. 6861–6871.
[27]
E. Rossi, F. Frasca, B. Chamberlain, D. Eynard, M. Bronstein, and F. Monti, SIGN: Scalable inception graph neural networks, arXiv preprint arXiv: 2004.11198, 2020.
[28]
C. Sun and G. Wu, Scalable and Adaptive graph neural networks with self-label-enhanced training, arXiv preprint arXiv: 2104.09376, 2021.
[29]
W. Zhang, Z. Yin, Z. Sheng, Y. Li, W. Ouyang, X. Li, Y. Tao, Z. Yang, and B. Cui, Graph attention multi-layer perceptron, in Proc. 28th ACM SIGKDD Conf. Knowledge Discovery and Data Mining, Washington, DC, USA, 2022, pp. 4560–4570.
Crossref
[30]

H. Chen, B. Perozzi, Y. Hu, and S. Skiena, HARP: Hierarchical representation learning for networks, Proc. AAAI Conf. Artif. Intell., vol. 32, no. 1, pp. 2127–2134, 2018.
Crossref Google Scholar
[31]
G. Fu, C. Hou, and X. Yao, Learning topological representation for networks via hierarchical sampling, in Proc. Int. Joint Conf. Neural Networks (IJCNN ), Budapest, Hungary, 2019, pp. 1–8.
Crossref
[32]
A. K. Bhowmick, K. Meneni, M. Danisch, J. L. Guillaume, and B. Mitra, LouvainNE: Hierarchical Louvain method for high quality and scalable network embedding, in Proc. 13th Int. Conf. Web Search and Data Mining, Houston, TX, USA, 2020, pp. 43–51.
Crossref
[33]

E. Ranjan, S. Sanyal, and P. Talukdar, ASAP: Adaptive structure aware pooling for learning hierarchical graph representations, Proc. AAAI Conf. Artif. Intell., vol. 34, no. 4, pp. 5470–5477, 2020.
Crossref Google Scholar
[34]
R. Ying, J. You, C. Morris, X. Ren, W. L. Hamilton, and J. Leskovec, Hierarchical graph representation learning with differentiable pooling, in Proc. 32nd Int. Conf. Neural Information Processing Systems, Montréal, Canada, 2018, pp. 4805–4815.
[35]

V. D. Blondel, J. L. Guillaume, R. Lambiotte, and E. Lefebvre, Fast unfolding of communities in large networks, J. Stat. Mech., vol. 2008, no. 10, p. P10008, 2008.
Crossref Google Scholar
[36]

M. E. J. Newman and M. Girvan, Finding and evaluating community structure in networks, Phys. Rev. E, vol. 69, no. 2, p. 026113, 2004.
Crossref Google Scholar
[37]

M. E. J. Newman, Modularity and community structure in networks, Proc. Natl. Acad. Sci. U. S. A., vol. 103, no. 23, pp. 8577–8582, 2006.
Crossref Google Scholar
[38]
B. Hu, Y. Fang, and C. Shi, Adversarial learning on heterogeneous information networks, in Proc. 25th ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining, Anchorage, AK, USA, 2019, pp. 120–129.
Crossref
[39]
X. Li, D. Ding, B. Kao, Y. Sun, and N. Mamoulis, Leveraging meta-path contexts for classification in heterogeneous information networks, in Proc. IEEE 37th Int. Conf. Data Engineering (ICDE ), Chania, Greece, 2021, pp. 912–923.
Crossref
[40]
K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor, Freebase: A collaboratively created graph database for structuring human knowledge, in Proc. 2008 ACM SIGMOD Int. Conf. Management of data, Vancouver, Canada, 2008, pp. 1247–1250.
Crossref
[41]

H. Wan, Y. Zhang, J. Zhang, and J. Tang, AMiner: Search and mining of academic social networks, Data Intell., vol. 1, no. 1, pp. 58–76, 2019.
Crossref Google Scholar
[42]
A. Grover and J. Leskovec, node2vec: Scalable feature learning for networks, in Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, San Francisco, CA, USA, 2016, pp. 855–856.
Crossref
[43]
X. Wang, N. Liu, H. Han, and C. Shi, Self-supervised heterogeneous graph neural network with co-contrastive learning, in Proc. 27th ACM SIGKDD Conf. Knowledge Discovery & Data Mining, Virtual Event, Singapore, Singapore, 2021, pp. 1726–1736.
Crossref
[44]
T. Mikolov, K. Chen, G. Corrado, and J. Dean, Efficient estimation of word representations in vector space, arXiv preprint arXiv: 1301.3781, 2013.
Tsinghua Science and Technology
Volume 30 Issue 1,
February 2025
Pages 198-214
DOI: 10.26599/TST.2023.9010147
Cite this article:
Zhao H, Rui P, Chen J, et al. Restage: Relation Structure-Aware Hierarchical Heterogeneous Graph Embedding. Tsinghua Science and Technology, 2025, 30(1): 198-214. https://doi.org/10.26599/TST.2023.9010147

155

Views

10

Downloads

0

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 30 August 2023
Revised: 14 November 2023
Accepted: 06 December 2023
Published: 11 September 2024
© The Author(s) 2025.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return