AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Computer Science and Technology Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Regular Paper

Meta-Learning Based Few-Shot Link Prediction for Emerging Knowledge Graph

School of Computer Science and Technology, Soochow University, Suzhou 215000, China
Show Author Information

Abstract

Inductive knowledge graph embedding (KGE) aims to embed unseen entities in emerging knowledge graphs (KGs). The major recent studies of inductive KGE embed unseen entities by aggregating information from their neighboring entities and relations with graph neural networks (GNNs). However, these methods rely on the existing neighbors of unseen entities and suffer from two common problems: data sparsity and feature smoothing. Firstly, the data sparsity problem means unseen entities usually emerge with few triplets containing insufficient information. Secondly, the effectiveness of the features extracted from original KGs will degrade when repeatedly propagating these features to represent unseen entities in emerging KGs, which is termed feature smoothing problem. To tackle the two problems, we propose a novel model entitled Meta-Learning Based Memory Graph Convolutional Network (MMGCN) consisting of three different components: 1) the two-layer information transforming module (TITM) developed to effectively transform information from original KGs to emerging KGs; 2) the hyper-relation feature initializing module (HFIM) proposed to extract type-level features shared between KGs and obtain a coarse-grained representation for each entity with these features; and 3) the meta-learning training module (MTM) designed to simulate the few-shot emerging KGs and train the model in a meta-learning framework. The extensive experiments conducted on the few-shot link prediction task for emerging KGs demonstrate the superiority of our proposed model MMGCN compared with state-of-the-art methods.

Keywords

knowledge graphgraph representation learningfew-shot learninginductive link prediction

Electronic Supplementary Material

Download File(s)
JCST-2209-12863-Highlights.pdf (462.4 KB)

References

[1]
Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J. Freebase: A collaboratively created graph database for structuring human knowledge. In Proc. the 2008 ACM SIGMOD International Conference on Management of Data, Jun. 2008, pp.1247–1250. DOI: 10.1145/1376616.1376746.
Crossref
[2]
Carlson A, Betteridge J, Kisiel B, Settles B, Hruschka E, Mitchell T. Toward an architecture for never-ending language learning. In Proc. the 24th AAAI Conference on Artificial Intelligence, Jul. 2010, pp.1306–1313. DOI: 10.1609/AAAI.V24I1.7519.
Crossref
[3]
Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z. DBpedia: A nucleus for a Web of open data. In Proc. the 6th International Semantic Web Conference, 2nd Asian Semantic Web Conference, Nov. 2007, pp.722–735. DOI: 10.1007/978-3-540-76298-0_52.
Crossref
[4]
Wang Y X, Khan A, Wu T X, Jin J H, Yan H J. Semantic guided and response times bounded top-k similarity search over knowledge graphs. In Proc. the 36th IEEE International Conference on Data Engineering, Apr. 2020, pp.445–456. DOI: 10.1109/ICDE48307.2020.00045.
Crossref
[5]
Yang Z X. Biomedical information retrieval incorporating knowledge graph for explainable precision medicine. In Proc. the 43rd International ACM SIGIR conference on research and development in Information Retrieval, Jul. 2020, p.2486. DOI: 10.1145/3397271.3401458.
Crossref
[6]
Wong C M, Feng F, Zhang W, Vong C M, Chen H, Zhang Y C, He P, Chen H, Zhao K, Chen H J. Improving conversational recommender system by pretraining billion-scale knowledge graph. In Proc. the 37th IEEE International Conference on Data Engineering, Apr. 2021, pp.2607–2612. DOI: 10.1109/ICDE51399.2021.00291.
Crossref
[7]
Deng Z Y, Li C Y, Liu S J, Ali W, Shao J. Knowledge-aware group representation learning for group recommendation. In Proc. the 37th IEEE International Conference on Data Engineering, Apr. 2021, pp.1571–1582. DOI: 10.1109/ICDE51399.2021.00139.
Crossref
[8]
Hu S, Zou L, Yu J X, Wang H X, Zhao D Y. Answering natural language questions by subgraph matching over knowledge graphs (extended abstract). In Proc. the 34th IEEE International Conference on Data Engineering, Apr. 2018, pp.1815–1816. DOI: 10.1109/ICDE.2018.00265.
Crossref
[9]
Kaiser M, Roy R S, Weikum G. Reinforcement learning from reformulations in conversational question answering over knowledge graphs. In Proc. the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Jul. 2021, pp.459–469. DOI: 10.1145/3404835.3462859.
Crossref
[10]

Nickel M, Murphy K, Tresp V, Gabrilovich E. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE, 2016, 104(1): 11–33. DOI: 10.1109/JPROC.2015.2483592.
Crossref Google Scholar
[11]
Bordes A, Usunier N, García-Durán A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. In Proc. the 26th Annual Conference on Neural Information Processing Systems, Dec. 2013, pp.2787–2795.
[12]
Yang B S, Yih W T, He X D, Gao J F, Deng L. Embedding entities and relations for learning and inference in knowledge bases. In Proc. the 3rd International Conference on Learning Representations, May 2015.
[13]
Shi B X, Weninger T. Open-world knowledge graph completion. In Proc. the 32nd AAAI Conference on Artificial Intelligence, Feb. 2018, pp.1957–1964. DOI: 10.1609/aaai.v32i1.11535.
Crossref
[14]
Hamaguchi T, Oiwa H, Shimbo M, Matsumoto Y. Knowledge transfer for out-of-knowledge-base entities: A graph neural network approach. In Proc. the 26th International Joint Conference on Artificial Intelligence, Aug. 2017, pp.1802–1808. DOI: 10.24963/ijcai.2017/250.
Crossref
[15]
Wang P F, Han J L, Li C L, Pan R. Logic attention based neighborhood aggregation for inductive knowledge graph embedding. In Proc. the 33rd AAAI Conference on Artificial Intelligence, Jan. 27–Feb. 1, 2019, pp.7152–7159. DOI: 10.1609/aaai.v33i01.33017152.
Crossref
[16]
Chen M Y, Zhang W, Zhang W, Chen Q, Chen H J. Meta relational learning for few-shot link prediction in knowledge graphs. In Proc. the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Nov. 2019, pp.4217–4226. DOI: 10.18653/V1/D19-1431.
Crossref
[17]

He Z L, Chen P F, Li X Y, Wang Y F, Yu G B, Chen C L, Li X R, Zheng Z B. A spatiotemporal deep learning approach for unsupervised anomaly detection in cloud systems. IEEE Trans. Neural Networks and Learning Systems, 2023, 34(4): 1705–1719. DOI: 10.1109/TNNLS.2020.3027736.
Crossref Google Scholar
[18]
Zhang Y F, Wang W Q, Chen W, Xu J J, Liu A, Zhao L. Meta-learning based hyper-relation feature modeling for out-of-knowledge-base embedding. In Proc. the 30th ACM International Conference on Information and Knowledge Management, Oct. 2021, pp.2637–2646. DOI: 10.1145/3459637.3482367.
Crossref
[19]
Wang Z, Zhang J W, Feng J L, Chen Z. Knowledge graph embedding by translating on hyperplanes. In Proc. the 28th AAAI Conference on Artificial Intelligence, Jul. 2014, pp.1112–1119. DOI: 10.1609/AAAI.V28I1.8870.
Crossref
[20]
Sun Z Q, Deng Z H, Nie J Y, Tang J. RotatE: Knowledge graph embedding by relational rotation in complex space. In Proc. the 7th International Conference on Learning Representations, May 2019.
[21]
Zhang Z Q, Cai J Y, Zhang Y D, Wang J. Learning hierarchy-aware knowledge graph embeddings for link prediction. In Proc. the 34th AAAI Conference on Artificial Intelligence, Feb. 2020, pp.3065–3072. DOI: 10.1609/aaai.v34i03.5701.
Crossref
[22]
Nickel M, Tresp V, Kriegel H P. A three-way model for collective learning on multi-relational data. In Proc. the 28th International Conference on Machine Learning, Jun. 28–Jul. 1, 2011, pp.809–816.
[23]
Dettmers T, Minervini P, Stenetorp P, Riedel S. Convolutional 2D knowledge graph embeddings. In Proc. the 32nd AAAI Conference on Artificial Intelligence, Feb. 2018, pp.1811–1818. DOI: 10.1609/AAAI.V32I1.11573.
Crossref
[24]
Nguyen D Q, Nguyen T D, Nguyen D Q, Phung D. A novel embedding model for knowledge base completion based on convolutional neural network. In Proc. the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Jun. 2018, pp.327–333. DOI: 10.18653/v1/n18-2053.
Crossref
[25]
Schlichtkrull M, Kipf T N, Bloem P, van den Berg R, Titov I, Welling M. Modeling relational data with graph convolutional networks. In Proc. the 15th International Conference on Semantic Web, Jun. 2018, pp.593–607. DOI: 10.1007/978-3-319-93417-4_38.
Crossref
[26]
Shang C, Tang Y, Huang J, Bi J B, He X D, Zhou B W. End-to-end structure-aware convolutional networks for knowledge base completion. In Proc. the 33rd AAAI Conference on Artificial Intelligence, Jan. 27–Feb. 1, 2019, pp.3060–3067. DOI: 10.1609/AAAI.V33I01.33013060.
Crossref
[27]
Xiong W H, Yu M, Chang S Y, Guo X X, Wang W Y. One-shot relational learning for knowledge graphs. In Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, Oct. 31–Nov. 7, 2018, pp.1980–1990. DOI: 10.18653/V1/D18-1223.
Crossref
[28]
Zhang C X, Yao H X, Huang C, Jiang M, Li Z H, Chawla N V. Few-shot knowledge graph completion. In Proc. the 34th AAAI Conference on Artificial Intelligence, Feb. 2020, pp.3041–3048. DOI: 10.1609/AAAI.V34I03.5698.
Crossref
[29]

Muggleton S. Inductive logic programming. New Generation Computing, 1991, 8(4): 295–318. DOI: 10.1007/BF03037089.
Crossref Google Scholar
[30]
Yang F, Yang Z L, Cohen W W. Differentiable learning of logical rules for knowledge base reasoning. In Proc. the 31st Annual Conference on Neural Information Processing Systems, Dec. 2017, pp.2316–2325.
[31]
Cohen W W. TensorLog: A differentiable deductive database. arXiv: 1605.06523, 2016. https://arxiv.org/abs/1605.06523, Sept. 2024.
[32]
Sadeghian A, Armandpour M, Ding P, Wang D Z. DRUM: End-to-end differentiable rule mining on knowledge graphs. In Proc. the 33rd Annual Conference on Neural Information Processing Systems, Dec. 2019, Article No. 1375.
[33]
Qu M, Chen J K, Xhonneux L P A C, Bengio Y, Tang J. RNNlogic: Learning logic rules for reasoning on knowledge graphs. In Proc. the 9th International Conference on Learning Representations, May 2021.
[34]
Cheng K W, Liu J H, Wang W, Sun Y Z. RLogic: Recursive logical rule learning from knowledge graphs. In Proc. the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Aug. 2022, pp.179–189. DOI: 10.1145/3534678.3539421.
Crossref
[35]
Chen S Y, Fang H, Cai Y F, Huang X, Sun M M. Differentiable neuro-symbolic reasoning on large-scale knowledge graphs. In Proc. the 37th Annual Conference on Neural Information Processing Systems, Dec. 2023, Article No. 1222.
[36]
He Y Q, Wang Z H, Zhang P, Tu Z P, Ren Z C. VN network: Embedding newly emerging entities with virtual neighbors. In Proc. the 29th ACM International Conference on Information and Knowledge Management, Oct. 2020, pp.505–514. DOI: 10.1145/3340531.3411865.
Crossref
[37]
Baek J, Lee D B, Hwang S J. Learning to extrapolate knowledge: Transductive few-shot out-of-graph link prediction. In Proc. the 34th Annual Conference on Neural Information Processing Systems, Dec. 2020, Article No. 47.
[38]
Wang H W, Ren H Y, Leskovec J. Relational message passing for knowledge graph completion. In Proc. the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Aug. 2021, pp.1697–1707. DOI: 10.1145/3447548.3467247.
Crossref
[39]
Zhu Z C, Zhang Z B, Xhonneux L P, Tang J. Neural bellman-ford networks: A general graph neural network framework for link prediction. In Proc. the 35th Annual Conference on Neural Information Processing Systems, Dec. 2021, Article No. 2256.
[40]
Zhang Y Q, Yao Q M. Knowledge graph reasoning with relational digraph. In Proc. the 2022 ACM Web Conference, Apr. 2022, pp.912–924. DOI: 10.1145/3485447.3512008.
Crossref
[41]
Wang C J, Zhou X F, Pan S R, Dong L H, Song Z L, Sha Y. Exploring relational semantics for inductive knowledge graph completion. In Proc. the 36th AAAI Conference on Artificial Intelligence, Feb. 22–Mar. 1, 2022, pp.4184–4192. DOI: 10.1609/AAAI.V36I4.20337.
Crossref
[42]
Zhang Y Q, Zhou Z K, Yao Q M, Chu X W, Han B. AdaProp: Learning adaptive propagation for graph neural network based knowledge graph reasoning. In Proc. the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Aug. 2023, pp.3446–3457. DOI: 10.1145/3580305.3599404.
Crossref
[43]
Lee J, Chung C, Whang J J. InGram: Inductive knowledge graph embedding via relation graphs. In Proc. the 40th International Conference on Machine Learning, Jul. 2023, pp.18796–18809.
[44]
Teru K K, Denis E G, Hamilton W L. Inductive relation prediction by subgraph reasoning. In Proc. the 37th International Conference on Machine Learning, Jul. 2020, pp.9448–9457.
[45]
Chen J J, He H R, Wu F, Wang J. Topology-aware correlations between relations for inductive link prediction in knowledge graphs. In Proc. the 35th AAAI Conference on Artificial Intelligence, Feb. 2021, pp.6271–6278. DOI: 10.1609/AAAI.V35I7.16779.
Crossref
[46]
Liu S W, Grau B C, Horrocks I, Kostylev E V. INDIGO: GNN-based inductive knowledge graph completion using pair-wise encoding. In Proc. the 35th Annual Conference on Neural Information Processing Systems, Dec. 2021, Article No. 156.
[47]
Chen M Y, Zhang W, Zhu Y S, Zhou H T, Yuan Z G, Xu C L, Chen H J. Meta-knowledge transfer for inductive knowledge graph embedding. In Proc. the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Jul. 2022, pp.927–937. DOI: 10.1145/3477495.3531757.
Crossref
[48]
Xu X H, Zhang P, He Y Q, Chao C P, Yan C Y. Subgraph neighboring relations infomax for inductive link prediction on knowledge graphs. In Proc. the 31st International Joint Conference on Artificial Intelligence, Jul. 2022, pp.2341–2347. DOI: 10.24963/IJCAI.2022/325.
Crossref
[49]
Zhang Y F, Wang W Q, Yin H Z, Zhao P P, Chen W, Zhao L. Disconnected emerging knowledge graph oriented inductive link prediction. In Proc. the 39th IEEE International Conference on Data Engineering, Apr. 2023, pp.381–393. DOI: 10.1109/ICDE55515.2023.00036.
Crossref
[50]
Geng Y X, Chen J Y, Pan J Z, Chen M Y, Jiang S, Zhang W, Chen H J. Relational message passing for fully inductive knowledge graph completion. In Proc. the 39th IEEE International Conference on Data Engineering, Apr. 2023, pp.1221–1233. DOI: 10.1109/ICDE55515.2023.00098.
Crossref
[51]
Toutanova K, Chen D Q. Observed versus latent features for knowledge base and text inference. In Proc. the 3rd Workshop on Continuous Vector Space Models and their Compositionality, Jul. 2015, pp.57–66. DOI: 10.18653/V1/W15-4007.
Crossref
[52]
Xiong W H, Hoang T, Wang W Y. DeepPath: A reinforcement learning method for knowledge graph reasoning. In Proc. the 2017 Conference on Empirical Methods in Natural Language Processing, Sept. 2017, pp.564–573. DOI: 10.18653/v1/d17-1060.
Crossref
[53]
Bordes A, Weston J, Collobert R, Bengio Y. Learning structured embeddings of knowledge bases. In Proc. the 25th AAAI Conference on Artificial Intelligence, Aug. 2011, pp.301–306. DOI: 10.1609/AAAI.V25I1.7917.
Crossref
Journal of Computer Science and Technology
Volume 39 Issue 5,
October 2024
Pages 1058-1077
DOI: 10.1007/s11390-024-2863-8
Cite this article:
Zhang Y-F, Chen W, Zhao P-P, et al. Meta-Learning Based Few-Shot Link Prediction for Emerging Knowledge Graph. Journal of Computer Science and Technology, 2024, 39(5): 1058-1077. https://doi.org/10.1007/s11390-024-2863-8

95

Views

0

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 06 November 2022
Accepted: 24 April 2024
Published: 05 December 2024
© Institute of Computing Technology, Chinese Academy of Sciences 2024
Return