AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Article Link
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Regular Paper

Word-Pair Relevance Modeling with Multi-View Neural Attention Mechanism for Sentence Alignment

School of Computer Science and Technology, Soochow University, Suzhou 215006, China
Show Author Information

Abstract

Sentence alignment provides multi-lingual or cross-lingual natural language processing (NLP) applications with high-quality parallel sentence pairs. Normally, an aligned sentence pair contains multiple aligned words, which intuitively play different roles during sentence alignment. Inspired by this intuition, we propose to deal with the problem of sentence alignment by exploring the semantic interactionship among fine-grained word pairs within the framework of neural network. In particular, we first employ various relevance measures to capture various kinds of semantic interactions among word pairs by using a word-pair relevance network, and then model their importance by using a multi-view attention network. Experimental results on both monotonic and non-monotonic bitexts show that our proposed approach significantly improves the performance of sentence alignment.

Electronic Supplementary Material

Download File(s)
jcst-35-3-617-Highlights.pdf (605 KB)

References

[1]
Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. In Proc. the 3rd International Conference on Learning Representations, May 2015.
[2]
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser L, Polosukhin I. Attention is all you need. In Proc. the 31st International Conference on Neural Information Processing Systems, December 2017, pp.5998-6008.
[3]
Hermann K M, Blunsom P. Multilingual models for compositional distributed semantics. In Proc. the 52nd Annual Meeting of the Association for Computational Linguistics, June 2014, pp.58-68.
[4]
Nie J Y, Simard M, Isabelle P, Durand R. Cross-language information retrieval based on parallel texts and automatic mining of parallel texts from the web. In Proc. the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 1999, pp.74-81.
[5]
Martino G D S, Romeo S, Barrón-Cedeño A, Joty S, Màrquez L, Moschitti A, Nakov P. Cross-language question re-ranking. In Proc. the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, August 2017, pp.1145-1148.
[6]
Grégoire F, Langlais P. A deep neural network approach to parallel sentence extraction. arXiv: 1709.09783, 2017. https://arxiv.org/abs/1709.09783, September 2019.
[7]
Grover J, Mitra P. Bilingual word embeddings with bucketed CNN for parallel sentence extraction. In Proc. the 55th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, July 2017, pp.11-16.
[8]
Quan X, Kit C, Song Y. Non-monotonic sentence alignment via semisupervised learning. In Proc. the 51st Annual Meeting of the Association for Computational Linguistics, August 2013, pp.622-630.
[9]
Langlais P, Simard M, Véronis J. Methods and practical issues in evaluating alignment techniques. In Proc. the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics, January 1998, pp.711-717.
[10]
Indurkhya N, Damerau F J. Handbook of Natural Language Processing (2nd edition). Chapman and Hall/CRC, 2010.
[11]
Gale W A, Church K W. A program for aligning sentences in bilingual corpora. In Proc. the 29th Annual Meeting on Association for Computational Linguistics, June 1991, pp.177-184.
[12]
Ma X. Champollion: A robust parallel text sentence aligner. In Proc. the 5th International Conference on Language Resources and Evaluation, May 2006, pp.489-492.
[13]

Kit C,Webster J J, Sin K K, Pan H, Li H. Clause alignment for bilingual Hong Kong legal texts: A lexical-based approach. International Journal of Corpus Linguistics, 2004, 9(1): 29-52.

[14]
Sutskever I, Salakhutdinov R, Tenenbaum J B. Modelling relational data using Bayesian clustered tensor factorization. In Proc. the 23rd International Conference on Neural Information Processing Systems, December 2009, pp.1821-1828.
[15]
Jenatton R, Roux N L, Bordes A, Obozinski G. A latent factor model for highly multi-relational data. In Proc. the 26th International Conference on Neural Information Processing Systems, December 2012, pp.3167-3175.
[16]
Collobert R, Weston J. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proc. the 25th International Conference on Machine Learning, June 2008, pp.160-167.
[17]
Zou W Y, Socher R, Cer D, Manning C D. Bilingual word embeddings for phrase-based machine translation. In Proc. the 2013 Conference on Empirical Methods in Natural Language Processing, October 2013, pp.1393-1398.
[18]
Cho K, van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proc. the 2014 Conference on Empirical Methods in Natural Language Processing, October 2014, pp.1724-1734.
[19]
Zeiler M D. ADADELTA: An adaptive learning rate method. arXiv: 1212.5701, 2012. https://arxiv.org/abs/1212.5701, December 2019.
[20]
Lin Z, Feng M, dos Santos C N, Yu M, Xiang B, Zhou B, Bengio Y. A structured self-attentive sentence embedding. In Proc. the 5th International Conference on Learning Representations, April 2017.
[21]
Devlin J, Chang M W, Lee K, Toutanova K. BERT: Pretraining of deep bidirectional transformers for language understanding. In Proc. the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2019, pp.4171-4186.
[22]
Wu Y, Schuster M, Chen Z et al. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv: 1609.08144, 2016. https://arxiv.org/abs/1609.08144, September 2019.
[23]
Moore R C. Fast and accurate sentence alignment of bilingual corpora. In Proc. the 5th Conference of the Association for Machine Translation in the Americas, October 2002, pp.135-144.
[24]
Braune F, Fraser A. Improved unsupervised sentence alignment for symmetrical and asymmetrical parallel corpora. In Proc. the 23rd International Conference on Computational Linguistics: Poster Volume, August 2010, pp.81-89.
[25]
He H, Lin J. Pairwise word interaction modeling with deep neural networks for semantic similarity measurement. In Proc. the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2016, pp.937-948.
[26]
Wang Z, Hamza W, Florian R. Bilateral multi-perspective matching for natural language sentences. In Proc. the 26th International Joint Conference on Artificial Intelligence, August 2017, pp.4144-4150.
[27]
Seo M, Kembhavi A, Farhadi A, Hajishirzi H. Bidirectional attention flow for machine comprehension. In Proc. the 5th International Conference on Learning Representations, April 2017.
Journal of Computer Science and Technology
Pages 617-628
Cite this article:
Ding Y, Li J-H, Gong Z-X, et al. Word-Pair Relevance Modeling with Multi-View Neural Attention Mechanism for Sentence Alignment. Journal of Computer Science and Technology, 2020, 35(3): 617-628. https://doi.org/10.1007/s11390-020-9331-x

386

Views

3

Crossref

N/A

Web of Science

2

Scopus

0

CSCD

Altmetrics

Received: 26 December 2018
Revised: 04 November 2019
Published: 29 May 2020
©Institute of Computing Technology, Chinese Academy of Sciences 2020
Return