{"type":"ScholarlyArticle","title":"Effective Deep Memory Networks for Distant Supervised Relation Extraction","authors":[{"name":"Xiaocheng Feng","slug":"xiaocheng-feng","given":"Xiaocheng","family":"Feng","affiliation":"Harbin Institute of Technology , Harbin , China"},{"name":"Jiang Guo","slug":"jiang-guo","given":"Jiang","family":"Guo","affiliation":"Harbin Institute of Technology , Harbin , China"},{"name":"Bing Qin","slug":"bing-qin","given":"Bing","family":"Qin","affiliation":"Harbin Institute of Technology , Harbin , China"},{"name":"Ting Liu","slug":"ting-liu","given":"Ting","family":"Liu","affiliation":"Harbin Institute of Technology , Harbin , China"},{"name":"Yongjie Liu","slug":"yongjie-liu","given":"Yongjie","family":"Liu","affiliation":"Harbin Institute of Technology , Harbin , China"}],"abstract":"Distant supervised relation extraction (RE) has been an effective way of finding novel relational facts from text without labeled training data. Typically it can be formalized as a multi-instance multi-label problem.In this paper, we introduce a novel neural approach for distant supervised (RE) with specific focus on attention mechanisms.Unlike the feature-based logistic regression model and compositional neural models such as CNN, our approach includes two major attention-based memory components, which is capable of explicitly capturing the importance of each context word for modeling the representation of the entity pair, as well as the intrinsic dependencies between relations.Such importance degree and dependency relationship are calculated with multiple computational layers, each of which is a neural attention model over an external memory. Experiment on real-world datasets shows that our approach performs significantly and consistently better than various baselines.","doi":"10.24963/ijcai.2017/559","url":"https://ijcai.org/proceedings/2017/559","pdf":"https://ijcai.org/proceedings/2017/0559.pdf","markdown":"https://ijcai.org/proceedings/2017/559.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/559","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":559,"pages":{"first":4002,"last":4008},"keywords":["Natural Language Processing: Information Extraction","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Hoffmann et al., 2011] Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S Weld. Knowledge-based weak supervision for information ex- traction of overlapping relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 541-550. Association for Computational Linguis- tics, 2011.","title":"Knowledge-based weak supervision for information extraction of overlapping relations","authors":"Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, Daniel S Weld","year":"2011","venue":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Jiang et al., 2016] Xiaotian Jiang, Quan Wang, Peng Li, and Bin Wang. Relation extraction with multi-instance multi- label convolutional neural networks. pages 1471-1480, December 2016.","title":"Relation extraction with multi-instance multilabel convolutional neural networks","authors":"Xiaotian Jiang, Quan Wang, Peng Li, Bin Wang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Li et al., 2012] Qi Li, Haibo Li, Heng Ji, Wen Wang, Jing Zheng, and Fei Huang. Joint bilingual name tagging for parallel corpora. In Proceedings of the 21st ACM interna- tional conference on Information and knowledge manage- ment, pages 1727-1731. ACM, 2012.","title":"Joint bilingual name tagging for parallel corpora","authors":"Qi Li, Haibo Li, Heng Ji, Wen Wang, Jing Zheng, Fei Huang","year":"2012","venue":"Proceedings of the 21st ACM international conference on Information and knowledge management","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Lin et al., 2016] Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. Neural relation extrac- tion with selective attention over instances. In Proceedings of ACL, 2016.","title":"Neural relation extraction with selective attention over instances","authors":"Lin","year":"2016","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositional- ity. In Advances in neural information processing systems, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Mintz et al., 2009] Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. Distant supervision for relation extrac- tion without labeled data. In Proceedings of the Joint Con- ference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Pro- cessing of the AFNLP: Volume 2-Volume 2, pages 1003- 1011. Association for Computational Linguistics, 2009.","title":"Distant supervision for relation extraction without labeled data","authors":"Mike Mintz, Steven Bills, Rion Snow, Dan Jurafsky","year":"2009","venue":"Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Nguyen and Grishman, 2015a] Thien Huu Nguyen and Ralph Grishman. Event detection and domain adaptation with convolutional neural networks. Volume 2: Short Pa- pers, page 365, 2015.","title":"Event detection and domain adaptation with convolutional neural networks","authors":"Grishman ; Nguyen, Huu Thien, Ralph Nguyen, Grishman","year":"2015","venue":"Short Papers","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Nguyen and Grishman, 2015b] Thien Huu Nguyen and Ralph Grishman. Relation extraction: Perspective from convolutional neural networks. In Proceedings of NAACL- HLT, pages 39-48, 2015.","title":"Relation extraction: Perspective from convolutional neural networks","authors":"Grishman ; Nguyen, Huu Thien, Ralph Nguyen, Grishman","year":"2015","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In EMNLP, volume 14, pages 1532-43, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Riedel et al., 2010] Sebastian Riedel, Limin Yao, and An- drew McCallum. Modeling relations and their mentions without labeled text. In Joint European Conference on Ma- chine Learning and Knowledge Discovery in Databases, pages 148-163. Springer, 2010.","title":"Modeling relations and their mentions without labeled text","authors":"Sebastian Riedel, Limin Yao, Andrew Mccallum","year":"2010","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Rush et al., 2015] Alexander M Rush, Sumit Chopra, and Jason Weston. A neural attention model for ab- stractive sentence summarization. arXiv preprint arXiv:1509.00685, 2015.","title":"A neural attention model for abstractive sentence summarization","authors":"Sumit Alexander M Rush, Jason Chopra, Weston","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey E Hin- ton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Sukhbaatar et al., 2015] Sainbayar Sukhbaatar, Jason We- ston, Rob Fergus, et al. End-to-end memory networks. In Advances in neural information processing systems, pages 2440-2448, 2015.","title":"End-to-end memory networks","authors":"Sukhbaatar","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Sun et al., 2015] Yaming Sun, Lei Lin, Duyu Tang, Nan Yang, Zhenzhou Ji, and Xiaolong Wang. Modeling men- tion, context and entity with neural networks for entity dis- ambiguation. In IJCAI, pages 1333-1339, 2015.","title":"Modeling mention, context and entity with neural networks for entity disambiguation","authors":"Yaming Sun, Lei Lin, Duyu Tang, Nan Yang, Zhenzhou Ji, Xiaolong Wang","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/192"},{"position":15,"raw":"[Surdeanu et al., 2012] Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D Manning. Multi- instance multi-label learning for relation extraction. In Proceedings of the 2012 joint conference on empirical methods in natural language processing and computa- tional natural language learning, pages 455-465. Asso- ciation for Computational Linguistics, 2012.","title":"Multiinstance multi-label learning for relation extraction","authors":"Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, Christopher D Manning","year":"2012","venue":"Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tang et al., 2016] Duyu Tang, Bing Qin, and Ting Liu. As- pect level sentiment classification with deep memory net- work. arXiv preprint arXiv:1605.08900, 2016.","title":"Aspect level sentiment classification with deep memory network","authors":"Tang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Vu et al., 2016] Ngoc Thang Vu, Heike Adel, Pankaj Gupta, and Hinrich Schütze. Combining recurrent and convolu- tional neural networks for relation classification. In Pro- ceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies, pages 534-539, San Diego, California, June 2016. Association for Computa- tional Linguistics.","title":"Combining recurrent and convolutional neural networks for relation classification","authors":"Ngoc Thang Vu, Heike Adel, Pankaj Gupta, Hinrich Schütze","year":"2016","venue":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Weston et al., 2014] Jason Weston, Sumit Chopra, and An- toine Bordes. Memory networks. arXiv preprint arXiv:1410.3916, 2014.","title":"Memory networks","authors":"Jason Weston, Sumit Chopra","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xu et al., 2015] Yan Xu, Lili Mou, Ge Li, Yunchuan Chen, Hao Peng, and Zhi Jin. Classifying relations via long short term memory networks along shortest dependency paths. In Proceedings of Conference on Empirical Methods in Natural Language Processing (to appear), 2015.","title":"Classifying relations via long short term memory networks along shortest dependency paths","authors":"Yan Xu, Lili Mou, Ge Li, Yunchuan Chen, Hao Peng, Zhi Jin","year":"2015","venue":"Proceedings of Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zeiler, 2012] Matthew D Zeiler. Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701, 2012.","title":"Adadelta: an adaptive learning rate method","authors":"D Matthew, Zeiler","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zeng et al., 2014] Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao, et al. Relation classification via convolutional deep neural network. In COLING, pages 2335-2344, 2014.","title":"Relation classification via convolutional deep neural network","authors":"Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao","year":"2014","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zeng et al., 2015] Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. Distant supervision for relation extraction via piecewise convolutional neural networks. In EMNLP, pages 1753-1762, 2015.","title":"Distant supervision for relation extraction via piecewise convolutional neural networks","authors":"Daojian Zeng, Kang Liu, Yubo Chen, Jun Zhao","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2006] Min Zhang, Jie Zhang, Jian Su, and Guodong Zhou. A composite kernel to extract relations between entities with both flat and structured features. In COLING-ACL'2006, pages 825-832. Association for Computational Linguistics, 2006.","title":"A composite kernel to extract relations between entities with both flat and structured features","authors":"Zhang","year":"2006","venue":"COLING-ACL'2006","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}