AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Tsinghua University (Science and Technology) Article
PDF (5.2 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Publishing Language: Chinese

Chinese-oriented entity recognition method of character vocabulary combination sequence

Qingren WANG, Yinzi WANG, Hong ZHONG( ), Yiwen ZHANG
College of Computer Science and Technology, Anhui University, Hefei 230093, China
Show Author Information

Abstract

Objective

As the core task of information extraction, named entity recognition recognizes various types of named entities from the text. The task of Chinese-named entity recognition has benefited from the application of deep learning in character vocabulary representation, feature extraction, and other aspects, achieving rich results. However, this task still faces the challenge of a lack of vocabulary information, which has been regarded as one of the primary impediments to the development of a high-performance Chinese-named entity recognition (NER) system. While the automatically constructed dictionary contains rich lexical boundary information and lexical semantic information, the integration of word knowledge in the Chinese NER task still faces challenges, such as the effective integration of the semantic information of self-matching words and their context information into Chinese characters. Furthermore, although graph neural networks can be used to extract feature information from various Chinese character-vocabulary interaction diagrams in feature extraction, the challenge of how to fuse features based on the importance of the information from the respective interaction diagrams into the original input sequence is yet to be solved.

Methods

This paper proposes a Chinese-oriented entity recognition method of Chinese-vocabulary combination sequence. (1) First, this method proposes a Chinese-vocabulary combination sequence embedding structure that primarily uses self-matching words to replace the Chinese characters in the Chinese character sequence under consideration. To make complete use of the self-matching vocabulary information, we also constructed a sequence for the self-matching vocabulary and vectorized the vocabulary and Chinese characters. At the coding level, we obtained the context information of the Chinese character sequence, the vocabulary sequence, and the Chinese-word combination sequence using the BiLSTM model and then fused the information from the words in the Chinese word combination sequence into the corresponding words in the vocabulary sequence. Furthermore, the graph neural network was used to extract the features of different Chinese-vocabulary interaction diagrams so that the enhanced vocabulary information can be integrated into Chinese characters, which can not only make complete use of the vocabulary boundary information but also integrate the context information of the self-matching vocabulary sequence into characters while capturing the semantic information between the Chinese characters and words, further enriching the character features. Finally, the conditional random field was used to decode and label the entities. (2) Considering the importance of different Chinese character-word interaction diagram information to the original input Chinese character sequence is not the same, this method proposes a multigraph attention fusion structure. It assigns a score to the correlation of the Chinese character sequence based on different Chinese character-word interaction diagram information, differentiates between structural features based on their importance, and fuses different Chinese character-word interaction diagram information into the Chinese character sequence based on their proportions.

Results

The F1 value of the new method was higher than that of the original method on Weibo, Resume, OntoNotes4.0, and MSRA data by 3.17% (Weibo_all), 1.21%, 1.33%, and 0.43%, respectively, thus verifying the feasibility of the new method on Chinese NER tasks.

Conclusions

The experiment revealed that the proposed method is more effective than the original method.

Keywords

natural language processingnamed entity recognitiongraph attention neural networkcharacter-word combination embeddingmultigraph attention
CLC number: TP393.1 Document code: A Article ID: 1000-0054(2023)09-1326-13

References

[1]

JU S G, LI T N, SUN J P. Chinese fine-grained name entity recognition based on associated memory networks[J]. Journal of Software, 2021, 32(8): 2545-2556. (in Chinese)
Google Scholar
[2]
SUN J, GAO J F, ZHANG L, et al. Chinese named entity identification using class-based language model[C]// COLING 2002: The 19th International Conference on Computational Linguistics. Taipei, China: Association for Computational Linguistics, 2002: 1-7.
Crossref
[3]
BUNESCU R, MOONEY R. A shortest path dependency kernel for relation extraction[C]// Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing. Vancouver, Canada: Association for Computational Linguistics, 2005: 724-731.
Crossref
[4]

YE Y X, XUE H, WANG L, et al. Distant supervision neural network relation extraction base on noisy observation[J]. Journal of Software, 2020, 31(4): 1025-1038. (in Chinese)
Google Scholar
[5]
CHEN Y B, XU L H, LIU K, et al. Event extraction via dynamic multi-pooling convolutional neural networks[C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Beijing, China: Association for Computational Linguistics, 2015: 167-176.
Crossref
[6]

HE R F, DUAN S Y. Joint Chinese event extraction based multi-task learning[J]. Journal of Software, 2019, 30(4): 1015-1030. (in Chinese)
Google Scholar
[7]
MOLLÁ D, VAN ZAANEN M, SMITH D. Named entity recognition for question answering[C]// Proceedings of Australasian Language Technology Workshop 2006. Sydney, Australia: ALTA, 2006: 51-58.
[8]
BOSSELUT A, RASHKIN H, SAP M, et al. COMET: Commonsense transformers for automatic knowledge graph construction[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, 2019: 4762-4779.
Crossref
[9]

YANG D H, HE T, WANG H Z, et al. Survey on knowledge graph embedding learning[J]. Journal of Software, 2022, 33(9): 3370-3390. (in Chinese)
Google Scholar
[10]

WANG X, ZOU L, WANG C K, et al. Research on knowledge graph data management: A survey[J]. Journal of Software, 2019, 30(7): 2139-2174. (in Chinese)
Google Scholar
[11]
YANG J, TENG Z Y, ZHANG M S, et al. Combining discrete and neural features for sequence labeling[C]// 17th International Conference on Computational Linguistics and Intelligent Text Processing. Konya, Turkey: Springer, 2018: 140-154.
Crossref
[12]
HE H F, SUN X. A unified model for cross-domain and semi-supervised named entity recognition in Chinese social media[C]// Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco, USA: AAAI Press, 2017: 3216-3222.
Crossref
[13]
LI H B, HAGIWARA M, LI Q, et al. Comparison of the impact of word segmentation on name tagging for Chinese and Japanese[C]// Proceedings of the Ninth International Conference on Language Resources and Evaluation. Reykjavik, Iceland: LREC, 2014: 2532-2536.
[14]
HE J Z, WANG H F. Chinese named entity recognition and word segmentation based on character[C]// Proceedings of the Sixth SIGHAN Workshop on Chinese Language Processing. Hyderabad, India: ACL, 2008: 128-132.
[15]
ZHANG Y, YANG J. Chinese NER using lattice LSTM[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Melbourne, Australia: Association for Computational Linguistics, 2018: 1554-1564.
Crossref
[16]
SUI D B, CHEN Y B, LIU K, et al. Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics, 2019: 3830-3840.
Crossref
[17]

HU B, GENG T Y, DENG G, et al. Faster biomedical named entity recognition based on knowledge distillation[J]. Journal of Tsinghua University (Science and Technology), 2021, 61(9): 936-942. (in Chinese)
Google Scholar
[18]

TAN H Y, ZHENG J H, LIU K Y. Research on method of automatic recognition of Chinese place name based on transformation[J]. Journal of Software, 2001, 12(11): 1608-1613. (in Chinese)
Google Scholar
[19]

TSAI T H, WU S H, LEE C W, et al. Mencius: A Chinese named entity recognizer using the maximum entropy-based hybrid model[J]. IJCLCLP, 2004, 9(1): 65-82.
Google Scholar
[20]
ISOZAKI H, KAZAWA H. Efficient support vector classifiers for named entity recognition[C]// COLING 2002: The 19th International Conference on Computational Linguistics. Taipei, China: Association for Computational Linguistics, 2002: 1-7.
Crossref
[21]
BIKEL D M, MILLER S, SCHWARTZ R, et al. Nymble: A high-performance learning name-finder[C]// Proceedings of the Fifth Conference on Applied Natural Language Processing. Washington, USA: Association for Computational Linguistics, 1997: 194-201.
Crossref
[22]
LAFFERTY J D, MCCALLUM A, PEREIRA F C N. Conditional random fields: Probabilistic models for segmenting and labeling sequence data[C]// Proceedings of the Eighteenth International Conference on Machine Learning. San Francisco, USA: Morgan Kaufmann Publishers Inc., 2001: 282-289.
[23]

YIN X Z, ZHAO H, ZHAO J B, et al. Multi-neural network collaboration for Chinese military named entity recognition[J]. Journal of Tsinghua University (Science and Technology), 2020, 60(8): 648-655. (in Chinese)
Google Scholar
[24]
MA X Z, HOVY E. End-to-end sequence labeling via Bi-directional LSTM-CNNs-CRF[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Berlin, Germany: Association for Computational Linguistics, 2016: 1064-1074.
Crossref
[25]

CHIU J P C, NICHOLS E. Named entity recognition with bidirectional LSTM-CNNs[J]. Transactions of the Association for Computational Linguistics, 2016, 4: 357-370.
Crossref Google Scholar
[26]

COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011, 12: 2493-2537.
Google Scholar
[27]
YAN H, SUN Y, LI X N, et al. An embarrassingly easy but strong baseline for nested named entity recognition[J/OL]. (2022-09-15)[2023-03-20]. https://arxiv.org/abs/2208.04534.
[28]
LIU L Y, SHANG J B, REN X, et al. Empower sequence labeling with task-aware neural language model[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. New Orleans, USA: AAAI Press, 2018: 5253-5260.
Crossref
[29]
HUANG Z H, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging[J/OL]. (2015-08-09)[2023-03-20]. https://arxiv.org/abs/1508.01991.
[30]
LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition[C]// Proceedings of 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego, USA: Association for Computational Linguistics, 2016: 260-270.
Crossref
[31]
GUI T, MA R T, ZHANG Q, et al. CNN-based Chinese NER with lexicon rethinking[C]// Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Macao, China: IJCAI, 2019: 4982-4988.
Crossref
[32]
ZHANG Y, WALLACE B. A sensitivity analysis of (and practitioners' guide to) convolutional neural networks for sentence classification[C]// Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Taipei, China: Asian Federation of Natural Language Processing, 2017: 253-263.
[33]
LIU P F, QIU X P, HUANG X J. Recurrent neural network for text classification with multi-task learning[C]// Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. New York, USA: AAAI Press, 2016: 2873-2879.
[34]
LIU W, XU T G, XU Q H, et al. An encoding strategy based word-character LSTM for Chinese NER[C]// Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1(Long and Short Papers). Minneapolis, USA: Association for Computational Linguistics, 2019: 2379-2389.
[35]
CHO K, VAN MERRIËNBOER B, BAHDANAU D, et al. On the properties of neural machine translation: Encoder-decoder approaches[C]// 8th Workshop on Syntax, Semantics and Structure in Statistical Translation. Doha, Qatar: Association for Computational Linguistics, 2014: 103-111.
Crossref
[36]
STRUBELL E, VERGA P, BELANGER D, et al. Fast and accurate entity recognition with iterated dilated convolutions[C]// Proceedings of 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark: Association for Computational Linguistics, 2017: 2670-2680.
Crossref
[37]
VELI AČG KOVIĆG P, CUCURULL G, CASANOVA A, et al. Graph attention networks[C]// 6th International Conference on Learning Representations. Vancouver, Canada: ICLR, 2017.
[38]

GRAVES A. Supervised sequence labelling with recurrent neural networks[M]. Berlin, Germany: Springer, 2012.
Crossref
[39]
NIE Y Y, TIAN Y H, SONG Y, et al. Improving named entity recognition with attentive ensemble of syntactic Information[C]// Findings of the Association for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics, 2020: 4231-4245.
Crossref
[40]

VITERBI A. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm[J]. IEEE Transactions on Information Theory, 1967, 13(2): 260-269.
Crossref Google Scholar
[41]
WEISCHEDEL R, PALMER M, MARCUS M, et al. OntoNotes release 4.0[EB/OL]. (2011-02-15)[2023-03-20]. https://doi.org/10.35111/gfjf-7r50.
[42]
LEVOW G A. The third international Chinese language processing bakeoff: Word segmentation and named entity recognition[C]// Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing. Sydney, Australia: Association for Computational Linguistics, 2006: 108-117.
[43]
PENG N Y, DREDZE M. Named entity recognition for Chinese social media with jointly trained embeddings[C]// Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal: Association for Computational Linguistics, 2015: 548-554.
Crossref
[44]
LI S, ZHAO Z, HU R F, et al. Analogical reasoning on Chinese morphological and semantic relations[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Melbourne, Australia: Association for Computational Linguistics, 2018: 138-143.
Crossref
[45]

SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: A simple way to prevent neural networks from overfitting[J]. The Journal of Machine Learning Research, 2014, 15(1): 1929-1958.
Google Scholar
[46]
KINGMA D P, BA J. Adam: A method for stochastic optimization[C]// 3rd International Conference on Learning Representations. San Diego, USA: ICLR, 2014.
[47]
MA R T, PENG M L, ZHANG Q, et al. Simplify the usage of lexicon in Chinese NER[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2020: 5951-5960.
Crossref
[48]
VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: Curran Associates Inc., 2017: 6000-6010.
[49]
LI X N, YAN H, QIU X P, et al. FLAT: Chinese NER using flat-lattice transformer[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistic. Association for Computational Linguistics, 2020: 6836-6842.
Crossref
Journal of Tsinghua University (Science and Technology)
Volume 63 Issue 9,
September 2023
Pages 1326-1338
DOI: 10.16511/j.cnki.qhdxxb.2023.21.009
Cite this article:
WANG Q, WANG Y, ZHONG H, et al. Chinese-oriented entity recognition method of character vocabulary combination sequence. Journal of Tsinghua University (Science and Technology), 2023, 63(9): 1326-1338. https://doi.org/10.16511/j.cnki.qhdxxb.2023.21.009

160

Views

0

Downloads

0

Crossref

2

Scopus

1

CSCD

Altmetrics
Received: 20 March 2023
Published: 15 September 2023
© Journal of Tsinghua University (Science and Technology). All rights reserved.
Return