Home Big Data Mining and Analytics Article
PDF (2.7 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Open Access

SCoAMPS: Semi-Supervised Graph Contrastive Learning Based on Associative Memory Network and Pseudo-Label Similarity

State Grid Yangzhou Power Supply Company, Yangzhou 225100, China
State Grid Jiangsu Electric Power Co. Ltd., Nanjing 210000, China
School of Cyber Science and Engineering, Southeast University, Nanjing 211189, China
School of Computer Science and Engineering, Southeast University, Nanjing 211189, China
Show Author Information

Abstract

Graph data have extensive applications in various domains, including social networks, biological reaction networks, and molecular structures. Graph classification aims to predict the properties of entire graphs, playing a crucial role in many downstream applications. However, existing graph neural network methods require a large amount of labeled data during the training process. In real-world scenarios, the acquisition of labels is extremely costly, resulting in labeled samples typically accounting for only a small portion of all training data, which limits model performance. Current semi-supervised graph classification methods, such as those based on pseudo-labels and knowledge distillation, still face limitations in effectively utilizing unlabeled graph data and mitigating pseudo-label bias issues. To address these challenges, we propose a Semi-supervised graph Contrastive learning based on Associative Memory network and Pseudo-label Similarity (SCoAMPS). SCoAMPS integrates pseudo-labeling techniques with contrastive learning by generating contrastive views through multiple encoders, selecting positive and negative samples using pseudo-label similarity, and defining associative memory network to alleviate pseudo-label bias problems. Experimental results demonstrate that SCoAMPS achieves significant performance improvements on multiple public datasets.

Keywords

graph attribute predictionlabel sparsitysemi-supervised graph learningcontrastive learning

References

[1]
W. Song, Z. Xiao, Y. Wang, L. Charlin, M. Zhang, and J. Tang, Session-based social recommendation via dynamic graph attention networks, in Proc. 12 th ACM Int. Conf. Web Search and Data Mining, Melbourne, Australia, 2019, pp. 555–563.
[2]

G. Chen, S. Peng, R. Zeng, Z. Hu, L. Cao, Y. Zhou, Z. Ouyang, and X. Nie, p-Norm broad learning for negative emotion classification in social networks, Big Data Mining and Analytics, vol. 5, no. 3, pp. 245–256, 2022.

Crossref Google Scholar
[3]

G. A. Pavlopoulos, M. Secrier, C. N. Moschopoulos, T. G. Soldatos, S. Kossida, J. Aerts, R. Schneider, and P. G. Bagos, Using graph theory to analyze biological networks, BioData Min., vol. 4, p. 10, 2011.

Crossref Google Scholar
[4]
C. Ying, T. Cai, S. Luo, S. Zheng, G. Ke, D. He, Y. Shen, and T. Y. Liu, Do transformers really perform bad for graph representation? in Proc. 35 th Int. Conf. Neural Information Processing Systems, Virtual Event, 2021, p. 2212.
[5]
Z. Hao, C. Lu, Z. Huang, H. Wang, Z. Hu, Q. Liu, E. Chen, and C. Lee, ASGN: An active semi-supervised graph neural network for molecular property prediction, in Proc. 26 th ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining, Virtual Event, 2020, pp. 731–752.
[6]

R. Kojima, S. Ishida, M. Ohta, H. Iwata, T. Honma, and Y. Okuno, kGCN: A graph-based deep learning framework for chemical structures, J. Cheminform., vol. 12, no. 1, p. 32, 2020.

Crossref Google Scholar
[7]

A. D. Becke, Perspective: Fifty years of density-functional theory in chemical physics, J. Chem. Phys., vol. 140, no. 18, p. 18A301, 2014.

Crossref Google Scholar
[8]

W. Kohn and L. J. Sham, Self-consistent equations including exchange and correlation effects, Phys. Rev., vol. 140, no. 4A, pp. A1133–A1138, 1965.

Crossref Google Scholar
[9]

K. Hansen, F. Biegler, R. Ramakrishnan, W. Pronobis, O. A. von Lilienfeld, K. R. Müller, and A. Tkatchenko, Machine learning predictions of molecular properties: Accurate many-body potentials and nonlocality in chemical space, J. Phys. Chem. Lett., vol. 6, no. 12, pp. 2326–2331, 2015.

Crossref Google Scholar
[10]
C. Lu, Q. Liu, C. Wang, Z. Huang, P. Lin, and L. He, Molecular property prediction: A multilevel quantum interactions modeling perspective, in Proc. 33 rd AAAI Conf. Artificial Intelligence, Honolulu, HI, USA, 2019, pp. 1052–1060.
[11]
K. T. Schütt, P. J. Kindermans, H. E. Sauceda, S. Chmiela, A. Tkatchenko, and K. R. Müller, SchNet: A continuous-filter convolutional neural network for modeling quantum interactions, in Proc. 31 st Int. Conf. Neural Information Processing Systems, Long Beach, CA, USA, 2017, pp. 992–1002.
[12]
J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, Neural message passing for quantum chemistry, in Proc. 34 th Int. Conf. Machine Learning, Sydney, Australia, 2017, pp. 1263–1272.
[13]

J. Zhang and Q. Xu, Attention-aware heterogeneous graph neural network, Big Data Mining and Analytics, vol. 4, no. 4, pp. 233–241, 2021.

Crossref Google Scholar
[14]
T. N. Kipf and M. Welling, Semi-supervised classification with graph convolutional networks, in Proc. 5 th Int. Conf. Learning Representations, Toulon, France, 2017, doi: 10.48550/arXiv.1609.02907.
[15]
G. Zhong and C. M. Pun, Latent low-rank graph learning for multimodal clustering, in Proc. IEEE 37 th Int. Conf. Data Engineering, Chania, Greece, 2021, pp. 492–503.
[16]
S. Di, Q. Yao, Y. Zhang, and L. Chen, Efficient relation-aware scoring function search for knowledge graph embedding, in Proc. 37 th Int. Conf. Data Engineering, Chania, Greece, 2021, pp. 1104–1115.
[17]
Z. Wang, T. Xia, R. Jiang, X. Liu, K. S. Kim, X. Song, and R. Shibasaki, Forecasting ambulance demand with profiled human mobility via heterogeneous multi-graph neural networks, in Proc. IEEE 37 th Int. Conf. Data Engineering, Chania, Greece, 2021, pp. 1751–1762.
[18]
J. Li, Y. Rong, H. Cheng, H. Meng, W. Huang, and J. Huang, Semi-supervised graph classification: A hierarchical graph perspective, in Proc. World Wide Web Conf., San Francisco, CA, USA, 2019, pp. 972–982.
[19]
F. Y. Sun, J. Hoffmann, V. Verma, and J. Tang, InfoGraph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization, in Proc. 8 th Int. Conf. Learning Representations, Addis Ababa, Ethiopia, 2020, doi: 10.48550/arXiv.1908.01000.
[20]

Z. Meng, C. Chen, X. Zhang, W. Zhao, and X. Cui, Exploring fragment adding strategies to enhance molecule pretraining in AI-driven drug discovery, Big Data Mining and Analytics, vol. 7, no. 3, pp. 565–576, 2024.

Crossref Google Scholar
[21]
K. M. Borgwardt and H. P. Kriegel, Shortest-path kernels on graphs, in Proc. 15 th IEEE Int. Conf. Data Mining, Houston, TX, USA, 2005, p. 8.
[22]
N. Shervashidze, S. V. N. Vishwanathan, T. Petri, K. Mehlhorn, and K. M. Borgwardt, Efficient graphlet kernels for large graph comparison, in Proc. 12 th Int. Conf. Artificial Intelligence and Statistics, Beach, FL, USA, 2009, pp. 488–495.
[23]

N. Shervashidze, P. Schweitzer, E. J. van Leeuwen, K. Mehlhorn, and K. M. Borgwardt, Weisfeiler-lehman graph kernels, J. Mach. Learn. Res., vol. 12, pp. 2539–2561, 2011.

Google Scholar
[24]
M. Zhang, Z. Cui, M. Neumann, and Y. Chen, An end-to-end deep learning architecture for graph classification, in Proc. 32 nd AAAI Conf. Artificial Intelligence, New Orleans, LA, USA, 2018, pp. 4438–4445.
[25]
D. H. Lee, Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks, in Proc. Workshop on Challenges in Representation Learning, Atlanta, GA, USA, 2013, p. 896.
[26]
Y. Grandvalet and Y. Bengio, Semi-supervised learning by entropy minimization, in Proc. 17 th Int. Conf. Neural Information Processing Systems, Vancouver, Canada, 2004, pp. 529–536.
[27]
M. Sajjadi, M. Javanmardi, and T. Tasdizen, Regularization with stochastic transformations and perturbations for deep semi-supervised learning, in Proc. 30 th Int. Conf. Neural Information Processing Systems, Barcelona, Spain, 2016, pp. 1171–1179.
[28]
A. Tarvainen and H. Valpola, Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results, in Proc. 31 st Int. Conf. Neural Information Processing Systems, Long Beach, CA, USA, 2017, pp. 1195–1204.
[29]

J. Li, Y. Huang, H. Chang, and Y. Rong, Semi-supervised hierarchical graph classification, IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 5, pp. 6265–6276, 2023.

Google Scholar
[30]
W. Ju, J. Yang, M. Qu, W. Song, J. Shen, and M. Zhang, KGNN: Harnessing kernel-based networks for semi-supervised graph classification, in Proc. 15 th ACM Int. Conf. Web Search and Data Mining, Virtual Event, 2022, pp. 421–429.
[31]
X. Luo, W. Ju, M. Qu, C. Chen, M. Deng, X. S. Hua, and M. Zhang, DualGraph: Improving semi-supervised graph classification via dual contrastive learning, in Proc. 38 th Int. Conf. Data Engineering, Kuala Lumpur, Malaysia, 2022, pp. 699–712.
[32]
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, A simple framework for contrastive learning of visual representations, in Proc. 37 th Int. Conf. Machine Learning, Virtual Event, 2020, p. 149.
[33]
K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, Momentum contrast for unsupervised visual representation learning, in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Seattle, WA, USA, 2020, pp. 9729–9738.
[34]
P. Veličković, W. Fedus, W. L. Hamilton, P. Liò, Y. Bengio, and R. D. Hjelm, Deep graph infomax, arXiv preprint arXiv: 1809.10341 , 2019.
[35]
Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang, Graph contrastive learning with adaptive augmentation, in Proc. Web Conf. 2021, Ljubljana, Slovenia, 2021, pp. 2069–2080.
[36]
Y. Fang, Q. Zhang, H. Yang, X. Zhuang, S. Deng, W. Zhang, M. Qin, Z. Chen, X. Fan, and H. Chen, Molecular contrastive learning with chemical element knowledge graph, in Proc. 36 th AAAI Conf. Artificial Intelligence, Virtual Event, 2022, pp. 3968–3976.
[37]
Y. Yin, Q. Wang, S. Huang, H. Xiong, and X. Zhang, AutoGCL: Automated graph contrastive learning via learnable view generators, in Proc. 36 th AAAI Conf. Artificial Intelligence, Virtual Event, 2022, pp. 8892–8900.
[38]
N. Liu, X. Wang, D. Bo, C. Shi, and J. Pei, Revisiting graph contrastive learning from the perspective of graph spectrum, in Proc. 36 th Int. Conf. Neural Information Processing Systems, New Orleans, LA, USA, 2022, p. 215.
[39]
Q. Ji, J. Li, J. Hu, R. Wang, C. Zheng, and F. Xu, Rethinking dimensional rationale in graph contrastive learning from causal perspective, in Proc. 38 th AAAI Conf. Artificial Intelligence, Vancouver, Canada, 2024, pp. 12810–12820.
[40]
J. Li, Y. Jin, H. Gao, W. Qiang, C. Zheng, and F. Sun, Hierarchical topology isomorphism expertise embedded graph contrastive learning, in Proc. 38 th AAAI Conf. Artificial Intelligence, Vancouver, Canada, 2024, pp. 13518–13527.
[41]

Y. Dong, M. Luo, J. Li, Z. Liu, and Q. Zheng, Semi-supervised graph contrastive learning with virtual adversarial augmentation, IEEE Trans. Knowl. Data Eng., vol. 36, no. 8, pp. 4232–4244, 2024.

Crossref Google Scholar
[42]
Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang, and Y. Shen, Graph contrastive learning with augmentations, in Proc. 34 th Int. Conf. Neural Information Processing Systems, Vancouver, Canada, 2020, p. 488.
[43]
D. Berthelot, N. Carlini, E. D. Cubuk, A. Kurakin, K. Sohn, H. Zhang, and C. Raffel, ReMixMatch: Semi-supervised learning with distribution matching and augmentation anchoring, arXiv preprint arXiv: 1911.09785, 2020.
[44]
P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, Supervised contrastive learning, in Proc. 34 th Int. Conf. Neural Information Processing Systems, Vancouver, Canada, 2020, p. 1567.
[45]
W. Hu, M. Fey, M. Zitnik, Y. Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovec, Open graph benchmark: Datasets for machine learning on graphs, in Proc. 34 th Int. Conf. Neural Information Processing Systems, Vancouver, Canada, 2020, p. 1855.
[46]
C. Morris, N. M. Kriege, F. Bause, K. Kersting, P. Mutzel, and M. Neumann, TUDataset: A collection of benchmark datasets for learning with graphs, in Proc. ICML 2020 Workshop on Graph Representation Learning and Beyond, doi: 10.48550/arXiv.2007.08663.
[47]

T. Miyato, S. I. Maeda, M. Koyama, and S. Ishii, Virtual adversarial training: A regularization method for supervised and semi-supervised learning, IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 8, pp. 1979–1993, 2019.

Crossref Google Scholar
Big Data Mining and Analytics
Volume 8 Issue 2,
April 2025
Pages 273-291
DOI: 10.26599/BDMA.2024.9020060
Cite this article:
Gong Z, Chen S, Dai Q, et al. SCoAMPS: Semi-Supervised Graph Contrastive Learning Based on Associative Memory Network and Pseudo-Label Similarity. Big Data Mining and Analytics, 2025, 8(2): 273-291. https://doi.org/10.26599/BDMA.2024.9020060
Metrics & Citations  
Article History
Copyright
Rights and Permissions
Return