AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Big Data Mining and Analytics Article
PDF (1,020.3 KB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

KeyEE: Enhancing Low-Resource Generative Event Extraction with Auxiliary Keyword Sub-Prompt

Hunan Key Laboratory of Bioinformatics, School of Computer Science and Engineering, Central South University, Changsha 410083, China
Big Data Institute, Central South University, Changsha 410083, China
Show Author Information

Abstract

Event Extraction (EE) is a key task in information extraction, which requires high-quality annotated data that are often costly to obtain. Traditional classification-based methods suffer from low-resource scenarios due to the lack of label semantics and fine-grained annotations. While recent approaches have endeavored to address EE through a more data-efficient generative process, they often overlook event keywords, which are vital for EE. To tackle these challenges, we introduce KeyEE, a multi-prompt learning strategy that improves low-resource event extraction by Event Keywords Extraction(EKE). We suggest employing an auxiliary EKE sub-prompt and concurrently training both EE and EKE with a shared pre-trained language model. With the auxiliary sub-prompt, KeyEE learns event keywords knowledge implicitly, thereby reducing the dependence on annotated data. Furthermore, we investigate and analyze various EKE sub-prompt strategies to encourage further research in this area. Our experiments on benchmark datasets ACE2005 and ERE show that KeyEE achieves significant improvement in low-resource settings and sets new state-of-the-art results.

Keywords

natural language processingEvent Extraction (EE)Multi-Prompt Learning (MPL)low-resource

References

[1]

J. Yao, B. Zhang, C. Li, D. Hong, and J. Chanussot, Extended vision transformer (ExViT) for land use and land cover classification: A multimodal deep learning framework, IEEE Trans. Geosci. Remote. Sens., vol. 61, p. 5514415, 2023.
Crossref Google Scholar
[2]
C. Li, B. Zhang, D. Hong, J. Yao, and J. Chanussot, LRR-net: An interpretable deep unfolding network for hyperspectral anomaly detection, IEEE Trans. Geosci. Remote Sensing, vol. 61, pp. 1–12, 2023.
Crossref
[3]
X. Wu, D. Hong, and J. Chanussot, UIU-net: U-net in U-net for infrared small object detection, IEEE Trans. Image Process., vol. 32, pp. 364–376, 1809.
Crossref
[4]

X. Wu, D. Hong, and J. Chanussot, Convolutional neural networks for multimodal remote sensing data classification, IEEE Trans. Geosci. Remote. Sens., vol. 60, p. 5517010, 2021.
Crossref Google Scholar
[5]

D. Hong, N. Yokoya, J. Chanussot, and X. X. Zhu, An augmented linear mixing model to address spectral variability for hyperspectral unmixing, IEEE Trans. Image Process., vol. 28, no. 4, pp. 1923–1938, 2019.
Crossref Google Scholar
[6]
X. Du and C. Cardie, Event extraction by answering (almost) natural questions, in Proc. 2020 Conf. Empirical Methods in Natural Language Processing (EMNLP ), https://doi.org/10.48550/arXiv.2004.13625, 2023.
Crossref
[7]
F. Li, W. Peng, Y. Chen, Q. Wang, L. Pan, Y. Lyu, and Y. Zhu, Event extraction as multi-turn question answering, in Proc. Findings of the Association for Computational Linguistics (EMNLP 2020), Virtual Event, pp. 829–838.
Crossref
[8]
S. Wang, M. Yu, S. Chang, L. Sun, and L. Huang, Query and extract: Refining event extraction as type-oriented binary decoding, in Proc. Findings of the Association for Computational Linguistics (ACL 2022), Dublin, Ireland, 2022. pp. 169–182.
Crossref
[9]
S. Yang, D. Feng, L. Qiao, Z. Kan, and D. Li, Exploring pre-trained language models for event extraction and generation, in Proc. 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 2019, pp. 5284–5294.
Crossref
[10]
Y. Lin, H. Ji, F. Huang, and L. Wu, A joint neural model for information extraction with global features, in Proc. 58th Annual Meeting of the Association for Computational Linguistics, Virtual Event, 2020. pp. 7999–8009.
Crossref
[11]

T. M. Nguyen and T. H. Nguyen, One for all: Neural joint modeling of entities and events, Proc. AAAI Conf. Artif. Intell., vol. 33, no. 1, pp. 6851–6858, 2019.
Crossref Google Scholar
[12]
A. Ramponi, R. van der Goot, R. Lombardo, and B. Plank, Biomedical event extraction as sequence labeling, in Proc. 2020 Conf. Empirical Methods in Natural Language Processing (EMNLP), Virtual Event, 2020, pp. 5357–5367
Crossref
[13]
D. Wadden, U. Wennberg, Y. Luan, and H. Hajishirzi, Entity, relation, and event extraction with contextualized span representations, in Proc. 2019 Conf. Empirical Methods in Natural Language Processing and the 9th Int. Joint Conf. Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, 2019, pp. 5784–5789.
Crossref
[14]

Z. Zhou, X. Dong, R. Meng, M. Wang, H. Yan, K. Yu, and K.-K R. Choo, Generative steganography via auto-generation of semantic object contours, IEEE Trans. Inform. Forensic Secur., vol. 18, pp. 2751–2765, 2023.
Crossref Google Scholar
[15]
Y. Cao, Z. Zhou, C. Chakraborty, M. Wang, Q. M. Jonathan Wu, X. Sun, and K. Yu, Generative steganography based on long readable text generation, IEEE Trans. Comput. Soc. Syst. doi:10.1109/TCSS.2022.3174013.
Crossref
[16]

Z. Zhou, X. Dong, Z. Li, K. Yu, C. Ding, and Y. Yang, Spatio-temporal feature encoding for traffic accident detection in VANET environment, IEEE Trans. Intell. Transp. Syst., vol. 23, no. 10, pp. 19772–19781, 2022.
Crossref Google Scholar
[17]
I.-H. Hsu, K.-H. Huang, E. Boschee, S. Miller, P. Natarajan, K.-W. Chang, and N. Peng, DEGREE: A data-efficient generation-based event extraction model, in Proc. 2022 Conf. North American Chapter of the Association for Computational Linguistics : Human Language Technologies, Seattle, WA, USA, 2022, pp. 1890–1908.
Crossref
[18]
X. Liu, H. Huang, G. Shi, and B. Wang, Dynamic prefix-tuning for generative template-based event extraction, in Proc. 60th Annual Meeting of the Association for Computational Linguistics (Volume 1 : Long Papers), Dublin, Ireland, 2022, pp. 5216–5228.
Crossref
[19]
G. Paolini, B. Athiwaratkun, J. Krone, J. Ma, A. Achille, R. Anubhai, C. N. D. Santos, B. Xiang, and S. Soatto, Structured prediction as translation between augmented natural languages, in Proc. International Conference on Learning Representations, Virtual Event, 2021, pp. 1−26.
[20]
Y. Lu, Q. Liu, D. Dai, X. Xiao, H. Lin, X. Han, L. Sun, and H. Wu, Unified structure generation for universal information extraction, in Proc. 60th Annual Meeting of the Association for Computational Linguistics (Volume 1 : Long Papers), Dublin, Ireland, 2022, pp. 5755–5772.
Crossref
[21]
G. Doddington, A. Mitchell, M. Przybocki, L. Ramshaw, S. Strassel, and R. Weischedel, The automatic content extraction (ACE) program— tasks, data, and evaluation, in Proceedings of the Fourth International Conference on Language Resources and Evaluation, Lisbon, Portugal, 2004, pp. 1−4.
[22]
Z. Song, A. Bies, S. Strassel, T. Riese, J. Mott, J. Ellis, J. Wright, S. Kulick, N. Ryant, and X. Ma, From light to rich ERE: Annotation of entities, relations, and events, in Proc. the 3rd Workshop on EVENTS : Definition, Detection, Coreference, and Representation, Denver, CO, USA, 2015. pp. 89–98.
Crossref
[23]
H. Cao, J. Li, F. Su, F. Li, H. Fei, S. Wu, B. Li, L. Zhao, and D. Ji, OneEE: A one-stage framework for fast overlapping and nested event extraction, in Proceedings of the 29th International Conference on Computational Linguistics, Gyeongju, Republic of Korea, 2022, pp. 1953–1964.
[24]
Y. Lu, H. Lin, J. Xu, X. Han, J. Tang, A. Li, L. Sun, M. Liao, and S. Chen, Text2Event: Controllable sequence-to-structure generation for end-to-end event extraction, in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Virtual Event, 2021, pp. 2795–2806.
Crossref
[25]
S. Li, H. Ji, and J. Han, Document-level event argument extraction by conditional generation, in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics : Human Language Technologies, Virtual Event, 2021, pp. 894–908.
Crossref
[26]
X. L. Li and P. Liang, Prefix-tuning: Optimizing continuous prompts for generation, in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Virtual Event, 2021, pp. 4582–4597.
[27]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need, arXiv preprint arXiv: 1706.03762, 2017.
[28]

P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing, ACM Comput. Surv., vol. 55, no. 9, pp. 1–35, 2023.
Crossref Google Scholar
[29]

Z. Jiang, F. F. Xu, J. Araki, and G. Neubig, How can we know what language models know, Trans. Assoc. Comput. Linguist., vol. 8, pp. 423–438, 2020.
Crossref Google Scholar
[30]
T. Schick and H. Schütze, Exploiting cloze-questions for few-shot text classification and natural language inference, in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics : Main Volume, Virtual Event, 2021, pp. 255–269.
Crossref
[31]
T. Gao, A. Fisch, and D. Chen, Making pre-trained language models better few-shot learners, in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Virtual Event, 2021, pp. 3816–3830.
Crossref
[32]
S. Kumar and P. Talukdar, Reordering examples helps during priming-based few-shot learning, in Proc. Findings of the Association for Computational Linguistics : ACL-IJCNLP 2021, Virtual Event, 2021, pp. 4507−4518.
Crossref
[33]
X. Han, W. Zhao, N. Ding, Z. Liu, and M. Sun, PTR: Prompt tuning with rules for text classification, AI Open, vol. 3, pp. 182–192, 2022.
Crossref
[34]
L. Cui, Y. Wu, J. Liu, S. Yang, and Y. Zhang, Template-based named entity recognition using BART, in Proc. Findings of the Association for Computational Linguistics : ACLIJCNLP 2021, Virtual Event, 2021, pp. 1835–1845.
Crossref
[35]
M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension, in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Virtual Event, 2020, pp. 7871–7880.
Crossref
[36]
M. Matena, Y. Zhou, W. Li, and P. J. Liu, Exploring the limits of transfer learning with a unified text-to-text transformer, Journal of Machine Learning Research, vol. 21, no. 1, pp. 140:1−140:67, 2020.
[37]
E. Cohen and C. Beck, Empirical analysis of beam search performance degradation in neural sequence models, in Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA, 2019, pp. 1290–1299.
[38]
M. Zhang, Y. Su, Z. Meng, Z. Fu, and N. Collier, COFFEE: A contrastive oracle-free framework for event extraction, arXiv preprint arXiv: 2303.14452, 2023.
Crossref
[39]
T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al., Transformers: State-of-the-art natural language processing, in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing : System Demonstrations, Virtual Event, 2020, pp. 38–45.
Crossref
[40]
I. Loshchilov and F. Hutter, Decoupled weight decay regularization, in Proc. 7th International Conference on Learning Representations, New Orleans, LA, USA, 2019, pp. 1−18.
Big Data Mining and Analytics
Volume 7 Issue 2,
June 2024
Pages 547-560
DOI: 10.26599/BDMA.2023.9020036
Cite this article:
Duan J, Liao X, An Y, et al. KeyEE: Enhancing Low-Resource Generative Event Extraction with Auxiliary Keyword Sub-Prompt. Big Data Mining and Analytics, 2024, 7(2): 547-560. https://doi.org/10.26599/BDMA.2023.9020036

359

Views

39

Downloads

1

Crossref

1

Web of Science

1

Scopus

0

CSCD

Altmetrics
Received: 26 July 2023
Revised: 06 November 2023
Accepted: 28 November 2023
Published: 22 April 2024
© The author(s) 2023.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return