Home Journal of Xinjiang University(Natural Science Edition in Chinese and English) Article
PDF (6.6 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Abstract
Keywords
References
Show full outline
Hide outline

Hybrid Augmentation of Text Feature and Graph Node for Graph Convolutional Networks Text Classification

School of Information Science and Technology, Guangdong University of Foreign Studies, Guangzhou Guangdong 510006, China
Shandong Key Laboratory of Language Resources Development and Application, Ludong University, Yantai Shandong 264025, China
Center for Linguistics and Applied Linguistics, Guangdong University of Foreign Studies, Guangzhou Guangdong 510420, China
Show Author Information

Abstract

The work will improve the structure on the basis of the BertGCN model, not only using a new algorithm to construct the edges of the graph, but also combining a hybrid enhancement of text features and graph nodes. The method not only has some optimization in the edge structure, but also makes fuller use of the extended semantic information of the text in the form of text feature enhancement and graph-enhanced nodes, while retaining the original text features. Four public datasets, R8, R52, Ohsumed and MR which are commonly used, are used to verify the effectiveness of this method. The experimental results show that compared with the BertGCN model and other baselines, the accuracy evaluation metric of the method on the four text classification data sets has been improved to varying degrees.

Keywords

BM25+text feature enhancementgraph node enhancementpre-training modelgraph convolution networkstext classification
Article ID: 2096-7675(2024)01-0069-09

References

[1]
ZHANG X X, WANG M Y, HE H X, et al. Mining key-nodes of emergency information dissemination network based on sentiment analysis[J]. Journal of Xinjiang University (Natural Science Edition), 2015, 32(3): 336-341. (in Chinese)
Google Scholar
[2]
AISHAN W, WEI W L, ZAOKERE K. Sentiment analysis based on BiLSTM+Attention in sports field[J]. Journal of Xinjiang University (Natural Science Edition in Chinese and English), 2020, 37(2): 142-149. (in Chinese)
Google Scholar
[3]
KIM Y. Convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar. Stroudsburg, PA, USA: Association for Computational Linguistics, 2014: 1746-1751.
Crossref
[4]
LIU P F, QIU X P, HUANG X J. Recurrent neural network for text classification with multi-task learning[C]//Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. July 9-15, 2016, New York, NewYork, USA. ACM, 2016: 2873-2879.
[5]
CHENG J P, DONG L, LAPATA M. Long short-term memory-networks for machine reading[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin, Texas. Stroudsburg, PA, USA: Association for Computational Linguistics, 2016: 551-561.
Crossref
[6]
SUNDERMEYER M, SCHLÜTER R, NEY H. LSTM neural networks for language modeling[C]//Interspeech 2012. ISCA: ISCA, 2012: 194-197.
Crossref
[7]
WU Z H, PAN S R, CHEN F W, et al. A comprehensive survey on graph neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(1): 4-24.
Crossref Google Scholar
[8]
DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[EB/OL]. 2018: arXiv: 1810.04805. https://arxiv.org/abs/1810.04805.pdf.
Google Scholar
[9]
WEI J, ZOU K. EDA: Easy data augmentation techniques for boosting performance on text classification tasks[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China. Stroudsburg, PA, USA: Association for Computational Linguistics, 2019: 6382-6388.
Crossref
[10]
FENG S Y, GANGAL V, WEI J, et al. A survey of data augmentation approaches for NLP[C]//Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Online. Stroudsburg, PA, USA: Association for Computational Linguistics, 2021: 968-988.
Crossref
[11]
LIN Y X, MENG Y X, SUN X F, et al. BertGCN: Transductive text classification by combining GCN and BERT[EB/OL]. 2021: arXiv: 2105.05727. https://arxiv.org/abs/2105.05727.pdf.
Crossref Google Scholar
[12]
LOWD D, DOMINGOS P. Naive Bayes models for probability estimation[C]//Proceedings of the 22nd International Conference on Machine Learning–ICML'05. August 7-11, 2005. Bonn, Germany. ACM, 2005: 529-536.
Crossref
[13]
ZHAO P N, LAI L F. Analysis of KNN density estimation[J]. IEEE Transactions on Information Theory, 2022, 68(12): 7971-7995.
Crossref Google Scholar
[14]
SAGI O, ROKACH L. Approximating XGBoost with an interpretable decision tree[J]. Information Sciences, 2021, 572: 522-542.
Crossref Google Scholar
[15]
DING C, BAO T Y, HUANG H L. Quantum-inspired support vector machine[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(12): 7210-7222.
Crossref Google Scholar
[16]
YANG J D, NAN X Y, ZHA Q. A robust weighted least squares support vector regression for air input prediction[J]. Journal of Xinjiang University (Natural Science Edition in Chinese and English), 2022, 39(2): 189-196. (in Chinese)
Google Scholar
[17]
KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[J]. ArXiv e-Prints, 2016: arXiv: 1609.02907.
Google Scholar
[18]
XIONG J C, XIONG Z P, CHEN K X, et al. Graph neural networks for automated de novo drug design[J]. Drug Discovery Today, 2021, 26(6): 1382-1393.
Crossref Google Scholar
[19]
LI Y, QIAN B Y, ZHANG X L, et al. Graph neural network-based diagnosis prediction[J]. Big Data, 2020, 8(5): 379-390.
Crossref Google Scholar
[20]
ZHANG J N, SHI X J, XIE J Y, et al. GaAN: Gated attention networks for learning on large and spatiotemporal graphs[EB/OL]. 2018: arXiv: 1803.07294. https://arxiv.org/abs/1803.07294.pdf.
Google Scholar
[21]
WANG H W, ZHAO M, XIE X, et al. Knowledge graph convolutional networks for recommender systems[C]//WWW'19: The World Wide Web Conference. May 13-17, 2019, San Francisco, CA, USA. ACM, 2019: 3307-3313.
[22]
LYU Y H, ZHAI C X. When documents are very long, BM25 fails![C]//Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval. July 24-28, 2011, Beijing, China. ACM, 2011: 1103-1104.
[23]
TROTMAN A, PUURULA A, BURGESS B. Improvements to BM25 and language models examined[C]//Proceedings of the 2014 Australasian Document Computing Symposium. Melbourne, VIC, Australia. ACM, 2014: 58-65.
Crossref
[24]
LYU Y H, ZHAI C X. Lower-bounding term frequency normalization[C]//Proceedings of the 20th ACM International Conference on Information and Knowledge Management. October 24-28, 2011, Glasgow, Scotland, UK. ACM, 2011: 7-16.
Crossref
[25]
YAO L, MAO C S, LUO Y. Graph convolutional networks for text classification[EB/OL]. 2018: arXiv: 1809.05679. https://arxiv.org/abs/1809.05679.pdf.
Google Scholar
[26]
YANG X Q, LIU W Y. Maximal-semantics-augmented BertGCN for text classification[EB/OL]. International Journal of Asian Language Processing, 2023, https://doi.org/10.1142/S2717554523500169.
Crossref Google Scholar
[27]
WU F, ZHANG T Y, DE SOUZA A H JR, et al. Simplifying graph convolutional networks[EB/OL]. 2019: arXiv: 1902.07153. https://arxiv.org/abs/1902.07153.pdf.
Google Scholar
[28]
KINGMA D P, BA J. Adam: A method for stochastic optimization[EB/OL]. 2014: arXiv: 1412.6980. https://arxiv.org/abs/1412.6980.pdf.
Google Scholar
Journal of Xinjiang University(Natural Science Edition in Chinese and English)
Volume 41 Issue 1,
January 2024
Pages 69-77,109
DOI: 10.13568/j.cnki.651094.651316.2023.07.05.0004
Cite this article:
YANG X, LIU W. Hybrid Augmentation of Text Feature and Graph Node for Graph Convolutional Networks Text Classification. Journal of Xinjiang University(Natural Science Edition in Chinese and English), 2024, 41(1): 69-77,109. https://doi.org/10.13568/j.cnki.651094.651316.2023.07.05.0004
Metrics & Citations  
Article History
Copyright
Return