{"type":"ScholarlyArticle","title":"Transformable Convolutional Neural Network for Text Classification","authors":[{"name":"Liqiang Xiao","slug":"liqiang-xiao","given":"Liqiang","family":"Xiao","affiliation":"State Key Lab of Advanced Optical Communication System and Network , Shanghai Jiao Tong University; Artificial Intelligence Institute , Shanghai Jiao Tong University"},{"name":"Honglun Zhang","slug":"honglun-zhang","given":"Honglun","family":"Zhang","affiliation":"State Key Lab of Advanced Optical Communication System and Network , Shanghai Jiao Tong University; Artificial Intelligence Institute , Shanghai Jiao Tong University"},{"name":"Wenqing Chen","slug":"wenqing-chen","given":"Wenqing","family":"Chen","affiliation":"State Key Lab of Advanced Optical Communication System and Network , Shanghai Jiao Tong University; Artificial Intelligence Institute , Shanghai Jiao Tong University"},{"name":"Yongkun Wang","slug":"yongkun-wang","given":"Yongkun","family":"Wang","affiliation":"Network and Information Center , Shanghai Jiao Tong University"},{"name":"Yaohui Jin","slug":"yaohui-jin","given":"Yaohui","family":"Jin","affiliation":"State Key Lab of Advanced Optical Communication System and Network , Shanghai Jiao Tong University; Artificial Intelligence Institute , Shanghai Jiao Tong University"}],"abstract":"Convolutional neural networks (CNNs) have shown their promising performance for natural language processing tasks, which extract n-grams as features to represent the input. However, n-gram based CNNs are inherently limited to fixed geometric structure and cannot proactively adapt to the transformations of features. In this paper, we propose two modules to provide CNNs with the flexibility for complex features and the adaptability for transformation, namely, transformable convolution and transformable pooling. Our method fuses dynamic and static deviations to redistribute the sampling locations, which can capture both current and global transformations. Our modules can be easily integrated by other models to generate new transformable networks. We test proposed modules on two state-of-the-art models, and the results demonstrate that our modules can effectively adapt to the feature transformation in text classification.","doi":"10.24963/ijcai.2018/625","url":"https://ijcai.org/proceedings/2018/625","pdf":"https://ijcai.org/proceedings/2018/0625.pdf","markdown":"https://ijcai.org/proceedings/2018/625.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/625","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":625,"pages":{"first":4496,"last":4502},"keywords":["Natural Language Processing: Text Classification","Natural Language Processing: Sentiment Analysis and Text Mining","Machine Learning: Neural Networks","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Bay et al., 2006] Herbert Bay, Tinne Tuytelaars, and Luc J. Van Gool. SURF: speeded up robust features. In Com- puter Vision -ECCV 2006, 9th European Conference on Computer Vision, Graz, Austria, May 7-13, 2006, Pro- ceedings, Part I, pages 404-417, 2006.","title":"SURF: speeded up robust features","authors":"Herbert Bay, Tinne Tuytelaars, Luc J Van Gool","year":"2006","venue":"Computer Vision -ECCV 2006, 9th European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blitzer et al., 2007] John Blitzer, Mark Dredze, and Fer- nando Pereira. Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In ACL 2007, Proceedings of ACL, June 23-30, 2007, Prague, Czech Republic, pages 187-205, 2007.","title":"Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification","authors":"John Blitzer, Mark Dredze, Fernando Pereira","year":"2007","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dai et al., 2017] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. De- formable convolutional networks. CoRR, abs/1703.06211, 2017.","title":"Deformable convolutional networks","authors":"Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, Yichen Wei","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Duchi et al., 2011] John Duchi, Elad Hazan, and Yoram Singer. Adaptive Subgradient Methods for Online Learn- ing and Stochastic Optimization. JMLR.org, 2011.","title":"Adaptive Subgradient Methods for Online Learning and Stochastic Optimization","authors":"John Duchi, Elad Hazan, Yoram Singer","year":"2011","venue":"JMLR.org","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Graves, 2013] Alex Graves. Generating sequences with re- current neural networks. CoRR, abs/1308.0850, 2013.","title":"Generating sequences with recurrent neural networks","authors":"Alex Graves","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Holschneider et al., 1988] M. Holschneider, R. Kronland- Martinet, J. Morlet, and Ph. Tchamitchian. A real-time algorithm for signal analysis with the help of the wavelet transform. pages 286-297, 1988.","title":"A real-time algorithm for signal analysis with the help of the wavelet transform","authors":"M Holschneider, R Kronland-Martinet, J Morlet, Ph Tchamitchian","year":"1988","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jaderberg et al., 2015] Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial transformer networks. In Advances in Neural Informa- tion Processing Systems, December 7-12, 2015, Montreal, Quebec, Canada, pages 2017-2025, 2015.","title":"Spatial transformer networks","authors":"Jaderberg","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jeon and Kim, 2017] Yunho Jeon and Junmo Kim. Active convolution: Learning the shape of convolution for image classification. CoRR, abs/1703.09076, 2017.","title":"Active convolution: Learning the shape of convolution for image classification","authors":"Jeon, Kim ; ] Yunho, Jeon, Junmo Kim","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jia et al., 2012] Yangqing Jia, Chang Huang, and Trevor Darrell. Beyond spatial pyramids: Receptive field learn- ing for pooled image features. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, June 16-21, 2012, pages 3370-3377, 2012.","title":"Beyond spatial pyramids: Receptive field learning for pooled image features","authors":"Jia","year":"2012","venue":"2012 IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Józefowicz et al., 2015] Rafal Józefowicz, Wojciech Zaremba, and Ilya Sutskever. An empirical exploration of recurrent network architectures. In Proceedings of, ICML 2015, Lille, France, 6-11 July 2015, pages 2342-2350, 2015.","title":"An empirical exploration of recurrent network architectures","authors":"Rafal Józefowicz, Wojciech Zaremba, Ilya Sutskever","year":"2015","venue":"Proceedings of, ICML 2015","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kalchbrenner et al., 2014] Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. A convolutional neural network for modelling sentences. In Proceedings of ACL 2014, June 22-27, 2014, Baltimore, MD, USA, Volume 1: Long Papers, pages 655-665, 2014.","title":"A convolutional neural network for modelling sentences","authors":"Nal Kalchbrenner, Edward Grefenstette, Phil Blunsom","year":"2014","venue":"Proceedings of ACL 2014","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kim, 2014] Yoon Kim. Convolutional neural networks for sentence classification. In Proceedings of, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1746-1751, 2014.","title":"Convolutional neural networks for sentence classification","authors":"Yoon Kim","year":"2014","venue":"A meeting of SIGDAT, a Special Interest Group of the ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lazebnik et al., 2006] Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce. Beyond bags of features: Spatial pyra- mid matching for recognizing natural scene categories. In (CVPR 2006), 17-22 June 2006, New York, NY, USA, pages 2169-2178, 2006.","title":"Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories","authors":"Svetlana Lazebnik, Cordelia Schmid, Jean Ponce","year":"2006","venue":"CVPR 2006)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Le and Mikolov, 2014] Quoc V. Le and Tomas Mikolov. Distributed representations of sentences and documents. Computer Science, 4:1188-1196, 2014.","title":"Distributed representations of sentences and documents","authors":"V Quoc, Tomas Le, Mikolov","year":"2014","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li and Roth, 2002] Xin Li and Dan Roth. Learning question classifiers. Coling, 12(24):556-562, 2002.","title":"Learning question classifiers","authors":"Roth ; Li, Xin Li, Dan Roth","year":"2002","venue":"Coling","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2015] Xiaodong Liu, Jianfeng Gao, Xiaodong He, Li Deng, Kevin Duh, and Ye Yi Wang. Representation learning using multi-task deep neural networks for seman- tic classification and information retrieval. In In NAACL HLT, pages 912-921, 2015.","title":"Representation learning using multi-task deep neural networks for semantic classification and information retrieval","authors":"Xiaodong Liu, Jianfeng Gao, Xiaodong He, Li Deng, Kevin Duh, Ye Yi, Wang","year":"2015","venue":"NAACL HLT","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lowe, 1999] David G. Lowe. Object recognition from local scale-invariant features. In ICCV, pages 1150-1157, 1999.","title":"Object recognition from local scale-invariant features","authors":"G David, Lowe","year":"1999","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Maas et al., 2011] Andrew L Maas, Raymond E Daly, Pe- ter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Meeting of the Association for Computational Linguistics, pages 142-150, 2011.","title":"Learning word vectors for sentiment analysis","authors":"Raymond E Andrew L Maas, Peter T Daly, Dan Pham, Andrew Y Huang, Christopher Ng, Potts","year":"2011","venue":"Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. Distributed rep- resentations of words and phrases and their composition- ality. Advances in Neural Information Processing Systems, 26:3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pang et al., 2004] Pang, Bo, Lee, and Lillian. A sentimen- tal education: sentiment analysis using subjectivity sum- marization based on minimum cuts. Proceedings of ACL, pages 271-278, 2004.","title":"A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts","authors":"Bo Pang, Lee, Lillian","year":"2004","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rublee et al., 2011] Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary R. Bradski. ORB: an efficient alter- native to SIFT or SURF. In IEEE International Confer- ence on Computer Vision, ICCV 2011, Barcelona, Spain, November 6-13, 2011, pages 2564-2571, 2011.","title":"ORB: an efficient alternative to SIFT or SURF","authors":"Ethan Rublee, Vincent Rabaud, Kurt Konolige, Gary R Bradski","year":"2011","venue":"IEEE International Conference on Computer Vision, ICCV 2011","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shen et al., 2014] Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire Mesnil. A latent semantic model with convolutional-pooling structure for informa- tion retrieval. In Proceedings of CIKM 2014, Shanghai, China, November 3-7, 2014, pages 101-110, 2014.","title":"A latent semantic model with convolutional-pooling structure for information retrieval","authors":"Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, Grégoire Mesnil","year":"2014","venue":"Proceedings of CIKM 2014","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Socher et al., 2013] R. Socher, A. Perelygin, J. Y. Wu, J. Chuang, C. D. Manning, A. Y. Ng, and C. Potts. Re- cursive deep models for semantic compositionality over a sentiment treebank. 2013.","title":"Recursive deep models for semantic compositionality over a sentiment treebank","authors":"R Socher, A Perelygin, J Y Wu, J Chuang, C D Manning, A Y Ng, C Potts","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tai et al., 2015] Kai Sheng Tai, Richard Socher, and Christopher D. Manning. Improved semantic represen- tations from tree-structured long short-term memory net- works. Computer Science, 5(1):: 36., 2015.","title":"Improved semantic representations from tree-structured long short-term memory networks","authors":"Kai Sheng, Tai, Richard Socher, Christopher D Manning","year":"2015","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zeiler, 2012] Matthew D. Zeiler. Adadelta: An adaptive learning rate method. Computer Science, 2012.","title":"Adadelta: An adaptive learning rate method","authors":"D Matthew, Zeiler","year":"2012","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2016] Ye Zhang, Stephen Roller, and Byron C. Wallace. MGNC-CNN: A simple approach to exploiting multiple word embeddings for sentence classification. In NAACL HLT 2016, San Diego California, USA, June 12- 17, 2016, pages 1522-1527, 2016.","title":"MGNC-CNN: A simple approach to exploiting multiple word embeddings for sentence classification","authors":"Ye Zhang, Stephen Roller, Byron C Wallace","year":"2016","venue":"NAACL HLT 2016","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}