Multiway Attention Networks for Modeling Sentence Pairs

Multiway Attention Networks for Modeling Sentence Pairs

Chuanqi Tan, Furu Wei, Wenhui Wang, Weifeng Lv, Ming Zhou

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 4411-4417. https://doi.org/10.24963/ijcai.2018/613

Modeling sentence pairs plays the vital role for judging the relationship between two sentences, such as paraphrase identification, natural language inference, and answer sentence selection. Previous work achieves very promising results using neural networks with attention mechanism. In this paper, we propose the multiway attention networks which employ multiple attention functions to match sentence pairs under the matching-aggregation framework. Specifically, we design four attention functions to match words in corresponding sentences. Then, we aggregate the matching information from each function, and combine the information from all functions to obtain the final representation. Experimental results demonstrate that the proposed multiway attention networks improve the result on the Quora Question Pairs, SNLI, MultiNLI, and answer sentence selection task on the SQuAD dataset.
Keywords:
Natural Language Processing: Question Answering
Natural Language Processing: Text Classification