{"type":"ScholarlyArticle","title":"Dynamic Compositional Neural Networks over Tree Structure","authors":[{"name":"Pengfei Liu","slug":"pengfei-liu","given":"Pengfei","family":"Liu","affiliation":"Shanghai Key Laboratory of Intelligent Information Processing , Fudan University School of Computer Science , Fudan University 825 Zhangheng Road , Shanghai , China"},{"name":"Xipeng Qiu","slug":"xipeng-qiu","given":"Xipeng","family":"Qiu","affiliation":"Shanghai Key Laboratory of Intelligent Information Processing , Fudan University School of Computer Science , Fudan University 825 Zhangheng Road , Shanghai , China"},{"name":"Xuanjing Huang","slug":"xuanjing-huang","given":"Xuanjing","family":"Huang","affiliation":"Shanghai Key Laboratory of Intelligent Information Processing , Fudan University School of Computer Science , Fudan University 825 Zhangheng Road , Shanghai , China"}],"abstract":"Tree-structured neural networks have proven to be effective in learning semantic representations by exploitingsyntactic information. In spite of their success, most existing models suffer from the underfitting problem: they recursively use the same shared compositional function throughout the whole compositional process and lack expressive power due to inability to capture the richness of compositionality.In this paper, we address this issue by introducing the dynamic compositional neural networks over tree structure (DC-TreeNN), in which the compositional function is dynamically generated by a meta network.The role of meta-network is to capture the metaknowledge across the different compositional rules and formulate them. Experimental results on two typical tasks show the effectiveness of the proposed models.","doi":"10.24963/ijcai.2017/566","url":"https://ijcai.org/proceedings/2017/566","pdf":"https://ijcai.org/proceedings/2017/0566.pdf","markdown":"https://ijcai.org/proceedings/2017/566.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/566","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":566,"pages":{"first":4054,"last":4060},"keywords":["Natural Language Processing: Natural Language Processing","Natural Language Processing: Natural Language Semantics"],"videos":[],"references":[{"position":1,"raw":"[Bertinetto et al., 2016] Luca Bertinetto, João F Henriques, Jack Valmadre, Philip Torr, and Andrea Vedaldi. Learn- ing feed-forward one-shot learners. In Advances in NIPS, pages 523-531, 2016. (a) The behaviour of 21-st neuron for sentence \"She had everyone in stitches\" (b) The behaviour of 27-th neuron for sentence \"An airplane is taking off\" Figure 5: The two heat maps describe the behaviours of neurons z21 and z27 from DC-TreeNN.","title":"The behaviour of 21-st neuron for sentence \"She had everyone in stitches","authors":"Luca Bertinetto, F João, Jack Henriques, Philip Valmadre, Andrea Torr, Vedaldi","year":"2016","venue":"b) The behaviour of 27-th neuron for sentence \"An airplane is taking off\" Figure 5: The two heat maps describe the behaviours of neurons z21 and z27 from DC-TreeNN","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bowman et al., 2014] Samuel R Bowman, Christopher Potts, and Christopher D Manning. Recursive neural networks can learn logical semantics. arXiv preprint arXiv:1406.1827, 2014.","title":"Recursive neural networks can learn logical semantics","authors":"Christopher Samuel R Bowman, Christopher D Potts, Manning","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bowman et al., 2015] Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on EMNLP, 2015.","title":"A large annotated corpus for learning natural language inference","authors":"R Samuel, Gabor Bowman, Christopher Angeli, Christopher D Potts, Manning","year":"2015","venue":"Proceedings of the 2015 Conference on EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In Proceedings of EMNLP, 2014.","title":"Learning phrase representations using rnn encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Collobert et al., 2011] Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. The JMLR, 12:2493-2537, 2011.","title":"Natural language processing (almost) from scratch","authors":"Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel Kuksa","year":"2011","venue":"The JMLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[De Brabandere et al., 2016] Bert De Brabandere, Xu Jia, Tinne Tuytelaars, and Luc Van Gool. Dynamic filter net- works. In NIPS, 2016.","title":"Dynamic filter networks","authors":"Bert De Brabandere, Xu Jia, Tinne Tuytelaars, Luc Van Gool","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dong et al., 2014] Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. Adaptive recursive neural network for target-dependent twitter sentiment classifica- tion. In ACL (2), pages 49-54, 2014.","title":"Adaptive recursive neural network for target-dependent twitter sentiment classification","authors":"Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, Ke Xu","year":"2014","venue":"ACL (2)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Duchi et al., 2011] John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. The JMLR, 12:2121-2159, 2011.","title":"Adaptive subgradient methods for online learning and stochastic optimization","authors":"John Duchi, Elad Hazan, Yoram Singer","year":"2011","venue":"The JMLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ha et al., 2016] David Ha, Andrew Dai, and Quoc V Le. Hypernetworks. arXiv preprint arXiv:1609.09106, 2016.","title":"Hypernetworks","authors":"David Ha, Andrew Dai, Quoc V Le","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kalchbrenner et al., 2014] Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. A convolutional neural network for modelling sentences. In Proceedings of ACL, 2014.","title":"A convolutional neural network for modelling sentences","authors":"Nal Kalchbrenner, Edward Grefenstette, Phil Blunsom","year":"2014","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kim, 2014] Yoon Kim. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882, 2014.","title":"Convolutional neural networks for sentence classification","authors":"Yoon Kim","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Klein and Manning, 2003] Dan Klein and Christopher D Manning. Accurate unlexicalized parsing. In Proceedings of the 41st ACL, pages 423-430, 2003.","title":"Accurate unlexicalized parsing","authors":"Dan Klein, Christopher D Manning","year":"2003","venue":"Proceedings of the 41st ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li and Roth, 2002] Xin Li and Dan Roth. Learning ques- tion classifiers. In Proceedings of the 19th International Conference on Computational Linguistics, pages 556-562, 2002.","title":"Learning question classifiers","authors":"Roth ; Li, Xin Li, Dan Roth","year":"2002","venue":"Proceedings of the 19th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2015] PengFei Liu, Xipeng Qiu, Xinchi Chen, Shiyu Wu, and Xuanjing Huang. Multi-timescale LSTM for modelling sentences and documents. In Proceedings of the Conference on EMNLP, 2015.","title":"Multi-timescale LSTM for modelling sentences and documents","authors":"Liu","year":"2015","venue":"Proceedings of the Conference on EMNLP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2016a] Pengfe Liu, Xipeng Qiu, Jifan Chen, and Xuanjing Huang. Deep fusion LSTMs for text semantic matching. In Proceedings of ACL, 2016.","title":"Deep fusion LSTMs for text semantic matching","authors":"Liu","year":"2016","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2016b] Pengfei Liu, Xipeng Qiu, Yaqian Zhou, Jifan Chen, and Xuanjing Huang. Modelling interaction of sentence pair with coupled-lstms. In Proceedings of the 2016 Conference on EMNLP, November 2016.","title":"Modelling interaction of sentence pair with coupled-lstms","authors":"Liu","year":"2016","venue":"Proceedings of the 2016 Conference on EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Marelli et al., 2014] Marco Marelli, Luisa Bentivogli, Marco Baroni, Raffaella Bernardi, Stefano Menini, and Roberto Zamparelli. Semeval-2014 task 1: Evaluation of compositional distributional. SemEval-2014, 2014.","title":"Semeval-2014 task 1: Evaluation of compositional distributional","authors":"Marco Marelli, Luisa Bentivogli, Marco Baroni, Raffaella Bernardi, Stefano Menini, Roberto Zamparelli","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Naik and Mammone, 1992] Devang K Naik and RJ Mam- mone. Meta-neural networks that learn by learning. In Neural Networks, 1992. IJCNN., volume 1, pages 437- 442. IEEE, 1992.","title":"Meta-neural networks that learn by learning","authors":"K Devang, R J Naik, Mammone","year":"1992","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pang and Lee, 2004] Bo Pang and Lillian Lee. A sentimen- tal education: Sentiment analysis using subjectivity sum- marization based on minimum cuts. In Proceedings of ACL, 2004.","title":"A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts","authors":"Lee ; Pang, Bo Pang, Lillian Lee","year":"2004","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pang and Lee, 2005] Bo Pang and Lillian Lee. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the ACL, pages 115-124, 2005.","title":"Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales","authors":"Lee ; Pang, Bo Pang, Lillian Lee","year":"2005","venue":"Proceedings of the ACL","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. Proceedings of the EMNLP, 12:1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"Proceedings of the EMNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Pollack, 1990] Jordan B Pollack. Recursive distributed rep- resentations. Artificial Intelligence, 46(1):77-105, 1990.","title":"Recursive distributed representations","authors":"B Jordan, Pollack","year":"1990","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Pylkkänen and McElree, 2006] Liina Pylkkänen and Brian McElree. The syntax-semantics interface: On-line compo- sition of sentence meaning. Handbook of psycholinguis- tics, 2:537-577, 2006.","title":"The syntax-semantics interface: On-line composition of sentence meaning","authors":"Mcelree Pylkkänen, Liina Pylkkänen, Brian Mcelree","year":"2006","venue":"Handbook of psycholinguistics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Socher et al., 2012] Richard Socher, Brody Huval, Christo- pher D Manning, and Andrew Y Ng. Semantic composi- tionality through recursive matrix-vector spaces. In Pro- ceedings of EMNLP, pages 1201-1211, 2012.","title":"Semantic compositionality through recursive matrix-vector spaces","authors":"Richard Socher, Brody Huval, Christopher D Manning, Andrew Y Ng","year":"2012","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Socher et al., 2013a] Richard Socher, John Bauer, Christo- pher D Manning, and Andrew Y Ng. Parsing with compo- sitional vector grammars. In Proceedings of ACL, 2013.","title":"Parsing with compositional vector grammars","authors":"Socher","year":"2013","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Socher et al., 2013b] Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of EMNLP, 2013.","title":"Recursive deep models for semantic compositionality over a sentiment treebank","authors":"Socher","year":"2013","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tai et al., 2015] Kai Sheng Tai, Richard Socher, and Christopher D Manning. Improved semantic representa- tions from tree-structured long short-term memory net- works. arXiv preprint arXiv:1503.00075, 2015.","title":"Improved semantic representations from tree-structured long short-term memory networks","authors":"Kai Sheng, Tai, Richard Socher, Christopher D Manning","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Williams et al., 2015] Lowri Williams, Christian Bannister, Michael Arribas-Ayllon, Alun Preece, and Irena Spasić. The role of idioms in sentiment analysis. Expert Systems with Applications, 42(21):7375-7385, 2015.","title":"The role of idioms in sentiment analysis","authors":"Lowri Williams, Christian Bannister, Michael Arribas-Ayllon, Alun Preece, Irena Spasić","year":"2015","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhao et al., 2015] Han Zhao, Zhengdong Lu, and Pascal Poupart. Self-adaptive hierarchical sentence model. arXiv preprint arXiv:1504.05070, 2015.","title":"Self-adaptive hierarchical sentence model","authors":"Han Zhao, Zhengdong Lu, Pascal Poupart","year":"2015","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2015/571"},{"position":31,"raw":"[Zhu et al., 2015] Xiao-Dan Zhu, Parinaz Sobhani, and Hongyu Guo. Long short-term memory over recursive structures. In ICML, pages 1604-1612, 2015.","title":"Long short-term memory over recursive structures","authors":"Xiao-Dan Zhu, Parinaz Sobhani, Hongyu Guo","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}