{"type":"ScholarlyArticle","title":"Adaptive Semantic Compositionality for Sentence Modelling","authors":[{"name":"Pengfei Liu","slug":"pengfei-liu","given":"Pengfei","family":"Liu","affiliation":"Shanghai Key Laboratory of Intelligent Information Processing , Fudan University School of Computer Science , Fudan University 825 Zhangheng Road , Shanghai , China"},{"name":"Xipeng Qiu","slug":"xipeng-qiu","given":"Xipeng","family":"Qiu","affiliation":"Shanghai Key Laboratory of Intelligent Information Processing , Fudan University School of Computer Science , Fudan University 825 Zhangheng Road , Shanghai , China"},{"name":"Xuanjing Huang","slug":"xuanjing-huang","given":"Xuanjing","family":"Huang","affiliation":"Shanghai Key Laboratory of Intelligent Information Processing , Fudan University School of Computer Science , Fudan University 825 Zhangheng Road , Shanghai , China"}],"abstract":"Representing a sentence with a fixed vector has shown its effectiveness in various NLP tasks. Most of the existing methods are based on neural network, which recursively apply different composition functions to a sequence of word vectors thereby obtaining a sentence vector.A hypothesis behind these approaches is that the meaning of any phrase can be composed of the meanings of its constituents.However, many phrases, such as idioms, are apparently non-compositional.To address this problem, we introduce a parameterized compositional switch, which outputs a scalar to adaptively determine whether the meaning of a phrase should be composed of its two constituents.We evaluate our model on five datasets of sentiment classification and demonstrate its efficacy with qualitative and quantitative experimental analysis .","doi":"10.24963/ijcai.2017/567","url":"https://ijcai.org/proceedings/2017/567","pdf":"https://ijcai.org/proceedings/2017/0567.pdf","markdown":"https://ijcai.org/proceedings/2017/567.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/567","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":567,"pages":{"first":4061,"last":4067},"keywords":["Machine Learning: Deep Learning","Natural Language Processing: Natural Language Processing","Natural Language Processing: Natural Language Semantics"],"videos":[],"references":[{"position":1,"raw":"[Balahur et al., 2013] Alexandra Balahur, Ralf Steinberger, Mijail Kabadjov, Vanni Zavarella, Erik Van Der Goot, Matina Halkia, Bruno Pouliquen, and Jenya Belyaeva. Sentiment analysis in the news. arXiv preprint arXiv:1309.6202, 2013.","title":"Sentiment analysis in the news","authors":"Alexandra Balahur, Ralf Steinberger, Mijail Kabadjov, Vanni Zavarella, Erik Van Der, Matina Goot, Bruno Halkia, Jenya Pouliquen, Belyaeva","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bowman et al., 2016] Samuel R Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D Manning, and Christopher Potts. A fast unified model for parsing and sentence understanding. arXiv preprint arXiv:1603.06021, 2016.","title":"A fast unified model for parsing and sentence understanding","authors":"Jon Samuel R Bowman, Abhinav Gauthier, Raghav Rastogi, Christopher D Gupta, Christopher Manning, Potts","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Collobert et al., 2011] Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. The JMLR, 12:2493-2537, 2011.","title":"Natural language processing (almost) from scratch","authors":"Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel Kuksa","year":"2011","venue":"The JMLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Denil et al., 2014] Misha Denil, Alban Demiraj, Nal Kalch- brenner, Phil Blunsom, and Nando de Freitas. Mod- elling, visualising and summarising documents with a single convolutional neural network. arXiv preprint arXiv:1406.3830, 2014.","title":"Modelling, visualising and summarising documents with a single convolutional neural network","authors":"Misha Denil, Alban Demiraj, Nal Kalchbrenner, Phil Blunsom, Nando De Freitas","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong et al., 2014] Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. Adaptive recursive neural network for target-dependent twitter sentiment classifica- tion. In ACL (2), pages 49-54, 2014.","title":"Adaptive recursive neural network for target-dependent twitter sentiment classification","authors":"Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, Ke Xu","year":"2014","venue":"ACL (2)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Duchi et al., 2011] John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. The JMLR, 12:2121-2159, 2011.","title":"Adaptive subgradient methods for online learning and stochastic optimization","authors":"John Duchi, Elad Hazan, Yoram Singer","year":"2011","venue":"The JMLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hashimoto and Tsuruoka, 2016] Kazuma Hashimoto and Yoshimasa Tsuruoka. Adaptive joint learning of compo- sitional and non-compositional phrase embeddings. arXiv preprint arXiv:1603.06067, 2016.","title":"Adaptive joint learning of compositional and non-compositional phrase embeddings","authors":"Tsuruoka Hashimoto, Kazuma Hashimoto, Yoshimasa Tsuruoka","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hu et al., 2014] Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. Convolutional neural network architectures for matching natural language sentences. In Advances in Neural Information Processing Systems, 2014.","title":"Convolutional neural network architectures for matching natural language sentences","authors":"Baotian Hu, Zhengdong Lu, Hang Li, Qingcai Chen","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kalchbrenner et al., 2014] Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. A convolutional neural network for modelling sentences. In Proceedings of ACL, 2014.","title":"A convolutional neural network for modelling sentences","authors":"Nal Kalchbrenner, Edward Grefenstette, Phil Blunsom","year":"2014","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kartsaklis et al., 2012] Dimitri Kartsaklis, Mehrnoosh Sadrzadeh, and Stephen Pulman. A unified sentence space for categorical distributional-compositional semantics: Theory and experiments. In In Proceedings of COLING: Posters. Citeseer, 2012.","title":"A unified sentence space for categorical distributional-compositional semantics: Theory and experiments","authors":"Dimitri Kartsaklis, Mehrnoosh Sadrzadeh, Stephen Pulman","year":"2012","venue":"Proceedings of COLING: Posters","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kim, 2014] Yoon Kim. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882, 2014.","title":"Convolutional neural networks for sentence classification","authors":"Yoon Kim","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Klein and Manning, 2003] Dan Klein and Christopher D Manning. Accurate unlexicalized parsing. In Proceedings of the 41st Annual Meeting on Association for Computa- tional Linguistics-Volume 1, pages 423-430, 2003.","title":"Accurate unlexicalized parsing","authors":"Dan Klein, Christopher D Manning","year":"2003","venue":"Proceedings of the 41st Annual Meeting on Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2015] PengFei Liu, Xipeng Qiu, Xinchi Chen, Shiyu Wu, and Xuanjing Huang. Multi-timescale long short-term memory neural network for modelling sen- tences and documents. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2015.","title":"Multi-timescale long short-term memory neural network for modelling sentences and documents","authors":"Liu","year":"2015","venue":"Proceedings of the Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2016a] PengFei Liu, Xipeng Qiu, and Xuanjing Huang. Recurrent neural network for text classification with multi-task learning. In Proceedings of International Joint Conference on Artificial Intelligence, 2016.","title":"Recurrent neural network for text classification with multi-task learning","authors":"Liu","year":"2016","venue":"Proceedings of International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2016/408"},{"position":16,"raw":"[Liu et al., 2016b] PengFei Liu, Xipeng Qiu, and Xuanjing Huang. Syntax-based attention model for natural language inference. arXiv preprint arXiv:1607.06556, 2016.","title":"Syntax-based attention model for natural language inference","authors":"Liu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2017] Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. Dynamic compositional neural networks over tree structure. arXiv preprint arXiv:1705.04153, 2017.","title":"Dynamic compositional neural networks over tree structure","authors":"Pengfei Liu, Xipeng Qiu, Xuanjing Huang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2017/566"},{"position":18,"raw":"[Mikolov et al., 2010] Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur. Re- current neural network based language model. In INTER- SPEECH, 2010.","title":"Recurrent neural network based language model","authors":"Mikolov","year":"2010","venue":"INTER-SPEECH","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nunberg et al., 1994] Geoffrey Nunberg, Ivan A Sag, and Thomas Wasow. Idioms. Language, pages 491-538, 1994.","title":"Idioms. Language","authors":"Geoffrey Nunberg, Ivan A Sag, Thomas Wasow","year":"1994","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pang and Lee, 2004] Bo Pang and Lillian Lee. A sentimen- tal education: Sentiment analysis using subjectivity sum- marization based on minimum cuts. In Proceedings of ACL, 2004.","title":"A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts","authors":"Lee ; Pang, Bo Pang, Lillian Lee","year":"2004","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pang and Lee, 2005] Bo Pang and Lillian Lee. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd annual meeting on association for computational linguis- tics, pages 115-124. Association for Computational Lin- guistics, 2005.","title":"Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales","authors":"Lee ; Pang, Bo Pang, Lillian Lee","year":"2005","venue":"Proceedings of the 43rd annual meeting on association for computational linguistics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Socher et al., 2010] Richard Socher, Christopher D Man- ning, and Andrew Y Ng. Learning continuous phrase rep- resentations and syntactic parsing with recursive neural networks. In Proceedings of the NIPS-2010 Deep Learn- ing and Unsupervised Feature Learning Workshop, pages 1-9, 2010.","title":"Learning continuous phrase representations and syntactic parsing with recursive neural networks","authors":"Richard Socher, Christopher D Manning, Andrew Y Ng","year":"2010","venue":"Proceedings of the NIPS-2010 Deep Learning and Unsupervised Feature Learning Workshop","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Socher et al., 2011] Richard Socher, Jeffrey Pennington, Eric H Huang, Andrew Y Ng, and Christopher D Man- ning. Semi-supervised recursive autoencoders for predict- ing sentiment distributions. In Proceedings of EMNLP, 2011.","title":"Semi-supervised recursive autoencoders for predicting sentiment distributions","authors":"Richard Socher, Jeffrey Pennington, Eric H Huang, Andrew Y Ng, Christopher D Manning","year":"2011","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Socher et al., 2013] Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of EMNLP, 2013.","title":"Recursive deep models for semantic compositionality over a sentiment treebank","authors":"Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, Christopher Potts","year":"2013","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc VV Le. Sequence to sequence learning with neural networks. In Advances in NIPS, pages 3104-3112, 2014.","title":"Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"Advances in NIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tai et al., 2015] Kai Sheng Tai, Richard Socher, and Christopher D Manning. Improved semantic representa- tions from tree-structured long short-term memory net- works. arXiv preprint arXiv:1503.00075, 2015.","title":"Improved semantic representations from tree-structured long short-term memory networks","authors":"Kai Sheng, Tai, Richard Socher, Christopher D Manning","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Williams et al., 2015] Lowri Williams, Christian Bannister, Michael Arribas-Ayllon, Alun Preece, and Irena Spasić. The role of idioms in sentiment analysis. Expert Systems with Applications, 42(21):7375-7385, 2015.","title":"The role of idioms in sentiment analysis","authors":"Lowri Williams, Christian Bannister, Michael Arribas-Ayllon, Alun Preece, Irena Spasić","year":"2015","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}