{"type":"ScholarlyArticle","title":"Ensemble Neural Relation Extraction with Adaptive Boosting","authors":[{"name":"Dongdong Yang","slug":"dongdong-yang","given":"Dongdong","family":"Yang","affiliation":"Unversity of Southern California"},{"name":"Senzhang Wang","slug":"senzhang-wang","given":"Senzhang","family":"Wang","affiliation":"Nanjing University of Aeronautics and Astronautics"},{"name":"Zhoujun Li","slug":"zhoujun-li","given":"Zhoujun","family":"Li","affiliation":"Beihang University"}],"abstract":"Relation extraction has been widely studied to extract new relational facts from open corpus. Previous relation extraction methods are faced with the problem of wrong labels and noisy data, which substantially decrease the performance of the model. In this paper, we propose an ensemble neural network model - Adaptive Boosting LSTMs with Attention, to more effectively perform relation extraction. Specifically, our model first employs the recursive neural network LSTMs to embed each sentence. Then we import attention into LSTMs by considering that the words in a sentence do not contribute equally to the semantic meaning of the sentence. Next via adaptive boosting, we build strategically several such neural classifiers. By ensembling multiple such LSTM classifiers with adaptive boosting, we could build a more effective and robust joint ensemble neural networks based relation extractor. Experiment results on real dataset demonstrate the superior performance of the proposed model, improving F1-score by about 8% compared to the state-of-the-art models.","doi":"10.24963/ijcai.2018/630","url":"https://ijcai.org/proceedings/2018/630","pdf":"https://ijcai.org/proceedings/2018/0630.pdf","markdown":"https://ijcai.org/proceedings/2018/630.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/630","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":630,"pages":{"first":4532,"last":4538},"keywords":["Natural Language Processing: Natural Language Processing","Natural Language Processing: Information Extraction"],"videos":[],"references":[{"position":1,"raw":"[De Boer et al., 2005] Pieter-Tjerk De Boer, Dirk P Kroese, Shie Mannor, and Reuven Y Rubinstein. A tutorial on the cross-entropy method. Annals of operations research, 134(1):19-67, 2005.","title":"A tutorial on the cross-entropy method","authors":"De Boer","year":"2005","venue":"Annals of operations research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Freund et al., 1996] Yoav Freund, Robert E Schapire, et al. Experiments with a new boosting algorithm. In Proceed- ings of The International Conference on Machine Learn- ing, pages 148-156, 1996.","title":"Experiments with a new boosting algorithm","authors":"Yoav Freund, Robert E Schapire","year":"1996","venue":"Proceedings of The International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Hoffmann et al., 2011] Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S Weld. Knowledge-based weak supervision for information ex- traction of overlapping relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 541- 550, 2011.","title":"Knowledge-based weak supervision for information extraction of overlapping relations","authors":"Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, Daniel S Weld","year":"2011","venue":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Kim and Kang, 2010] Myoung-Jong Kim and Dae-Ki Kang. Ensemble with neural networks for bankruptcy prediction. Expert systems with applications, 37(4):3373- 3379, 2010.","title":"Ensemble with neural networks for bankruptcy prediction","authors":"Kang ; ] Myoung-Jong Kim, Dae-Ki Kim, Kang","year":"2010","venue":"Expert systems with applications","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Kingma and Ba, 2014] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"Diederik Kingma, Jimmy Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Li et al., 2017] Dandan Li, Shuzhen Yao, Senzhang Wang, and Ying Wang. Cross-program design space exploration by ensemble transfer learning. In Proceedings of the 36th International Conference on Computer-Aided De- sign, pages 201-208, 2017.","title":"Cross-program design space exploration by ensemble transfer learning","authors":"Dandan Li, Shuzhen Yao, Senzhang Wang, Ying Wang","year":"2017","venue":"Proceedings of the 36th International Conference on Computer-Aided Design","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Lin et al., 2016] Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. Neural relation extrac- tion with selective attention over instances. In Proceedings of The 54th Annual Meeting of the Association for Compu- tational Linguistics, 2016.","title":"Neural relation extraction with selective attention over instances","authors":"Lin","year":"2016","venue":"Proceedings of The 54th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu et al., 2016] Yang Liu, Chengjie Sun, Lei Lin, and Xi- aolong Wang. Learning natural language inference us- ing bidirectional lstm model and inner-attention. arXiv preprint arXiv:1605.09090, 2016.","title":"Learning natural language inference using bidirectional lstm model and inner-attention","authors":"Yang Liu, Chengjie Sun, Lei Lin, Xiaolong Wang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Mintz et al., 2009] Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. Distant supervision for relation extrac- tion without labeled data. In Proceedings of the Joint Con- ference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language, pages 1003-1011, 2009.","title":"Distant supervision for relation extraction without labeled data","authors":"Mike Mintz, Steven Bills, Rion Snow, Dan Jurafsky","year":"2009","venue":"Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Miwa and Bansal, 2016] Makoto Miwa and Mohit Bansal. End-to-end relation extraction using lstms on sequences and tree structures. arXiv preprint arXiv:1601.00770, 2016.","title":"End-to-end relation extraction using lstms on sequences and tree structures","authors":"Bansal Miwa, Makoto Miwa, Mohit Bansal","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Riedel et al., 2010] Sebastian Riedel, Limin Yao, and An- drew McCallum. Modeling relations and their mentions without labeled text. In Proceedings of Joint European Conference on Machine Learning and Knowledge Discov- ery in Databases, pages 148-163. Springer, 2010.","title":"Modeling relations and their mentions without labeled text","authors":"Riedel","year":"2010","venue":"Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Rokach, 2010] Lior Rokach. Ensemble-based classifiers. Artificial Intelligence Review, 33(1):1-39, 2010.","title":"Lior Rokach. Ensemble-based classifiers","authors":"Rokach","year":"2010","venue":"Artificial Intelligence Review","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Surdeanu et al., 2012] Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D Manning. Multi- instance multi-label learning for relation extraction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computa- tional Natural Language Learning, pages 455-465, 2012.","title":"Multiinstance multi-label learning for relation extraction","authors":"Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, Christopher D Manning","year":"2012","venue":"Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Xu et al., 2015] Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of The International Conference on Machine Learning, pages 2048-2057, 2015.","title":"Show, attend and tell: Neural image caption generation with visual attention","authors":"Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, Yoshua Bengio","year":"2015","venue":"Proceedings of The International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Yang et al., 2016] Zichao Yang, Diyi Yang, Chris Dyer, Xi- aodong He, Alex Smola, and Eduard Hovy. Hierarchical attention networks for document classification. In Pro- ceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016.","title":"Hierarchical attention networks for document classification","authors":"Yang","year":"2016","venue":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ye et al., 2017] Hai Ye, Wenhan Chao, and Zhunchen Luo. Jointly extracting relations with class ties via effective deep ranking. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017.","title":"Jointly extracting relations with class ties via effective deep ranking","authors":"Hai Ye, Wenhan Chao, Zhunchen Luo","year":"2017","venue":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Zeng et al., 2014] Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao, et al. Relation classification via convolutional deep neural network. In Proceedings of 24th International Conference on Computational Linguis- tics, pages 2335-2344, 2014.","title":"Relation classification via convolutional deep neural network","authors":"Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao","year":"2014","venue":"Proceedings of 24th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zeng et al., 2015] Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 17-21, 2015.","title":"Distant supervision for relation extraction via piecewise convolutional neural networks","authors":"Daojian Zeng, Kang Liu, Yubo Chen, Jun Zhao","year":"2015","venue":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhang et al., 2015] Shu Zhang, Dequan Zheng, Xinchen Hu, and Ming Yang. Bidirectional long short-term mem- ory networks for relation classification. In Proceedings of the 29th Pacific Asia Conference on Language, Informa- tion and Computation, pages 73-78, 2015.","title":"Bidirectional long short-term memory networks for relation classification","authors":"Shu Zhang, Dequan Zheng, Xinchen Hu, Ming Yang","year":"2015","venue":"Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhang et al., 2017] Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D Manning. Position-aware attention and supervised data improve slot filling. In Proceedings of the 2017 Conference on Em- pirical Methods in Natural Language Processing, pages 35-45, 2017.","title":"Position-aware attention and supervised data improve slot filling","authors":"Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, Christopher D Manning","year":"2017","venue":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zheng et al., 2016] Hao Zheng, Zhoujun Li, Senzhang Wang, Zhao Yan, and Jianshe Zhou. Aggregating inter- sentence information to enhance relation extraction. In Proceedings of the 30th AAAI Conference on Artificial In- telligence, pages 3108-3115, 2016.","title":"Aggregating intersentence information to enhance relation extraction","authors":"Zhoujun Hao Zheng, Senzhang Li, Zhao Wang, Jianshe Yan, Zhou","year":"2016","venue":"Proceedings of the 30th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}