{"type":"ScholarlyArticle","title":"A Feature-Enriched Neural Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging","authors":[{"name":"Xinchi Chen","slug":"xinchi-chen","given":"Xinchi","family":"Chen","affiliation":"Shanghai Key Laboratory of Intelligent Information Processing , Fudan University School of Computer Science , Fudan University 825 Zhangheng Road , Shanghai , China"},{"name":"Xipeng Qiu","slug":"xipeng-qiu","given":"Xipeng","family":"Qiu","affiliation":"Shanghai Key Laboratory of Intelligent Information Processing , Fudan University School of Computer Science , Fudan University 825 Zhangheng Road , Shanghai , China"},{"name":"Xuanjing Huang","slug":"xuanjing-huang","given":"Xuanjing","family":"Huang","affiliation":"Shanghai Key Laboratory of Intelligent Information Processing , Fudan University School of Computer Science , Fudan University 825 Zhangheng Road , Shanghai , China"}],"abstract":"Recently, neural network models for natural language processing tasks have been increasingly focused on for their ability of alleviating the burden of manual feature engineering. However, the previous neural models cannot extract the complicated feature compositions as the traditional methods with discrete features. In this work, we propose a feature-enriched neural model for joint Chinese word segmentation and part-of-speech tagging task. Specifically, to simulate the feature templates of traditional discrete feature based models, we use different filters to model the complex compositional features with convolutional and pooling layer, and then utilize long distance dependency information with recurrent layer. Experimental results on five different datasets show the effectiveness of our proposed model.","doi":"10.24963/ijcai.2017/553","url":"https://ijcai.org/proceedings/2017/553","pdf":"https://ijcai.org/proceedings/2017/0553.pdf","markdown":"https://ijcai.org/proceedings/2017/553.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/553","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":553,"pages":{"first":3960,"last":3966},"keywords":["Natural Language Processing: Tagging, chunking, syntax, and parsing","Natural Language Processing: NLP Applications and Tools","Natural Language Processing: Phonology, Morphology, and word segmentation"],"videos":[],"references":[{"position":1,"raw":"[Chen et al., 2015a] Xinchi Chen, Xipeng Qiu, Chenxi Zhu, and Xuanjing Huang. Gated recursive neural network for chinese word segmentation. In ACL, 2015.","title":"Gated recursive neural network for chinese word segmentation","authors":"Chen","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2015b] Xinchi Chen, Xipeng Qiu, Chenxi Zhu, Pengfei Liu, and Xuanjing Huang. Long short-term mem- ory neural networks for chinese word segmentation. In EMNLP, pages 1197-1206, 2015.","title":"Long short-term memory neural networks for chinese word segmentation","authors":"Chen","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2017] Xinchi Chen, Zhan Shi, Xipeng Qiu, and Xuanjing Huang. Adversarial multi-criteria learning for chinese word segmentation. In ACL, 2017.","title":"Adversarial multi-criteria learning for chinese word segmentation","authors":"Chen","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Collobert et al., 2011] Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493-2537, 2011.","title":"Natural language processing (almost) from scratch","authors":"Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel Kuksa","year":"2011","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Elman, 1990] Jeffrey L Elman. Finding structure in time. Cognitive science, 14(2):179-211, 1990.","title":"Finding structure in time","authors":"Jeffrey L Elman","year":"1990","venue":"Cognitive science","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Forney Jr, 1973] G David Forney Jr. The viterbi algorithm. Proceedings of the IEEE, 61(3):268-278, 1973.","title":"The viterbi algorithm","authors":"David Forney","year":"1973","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2015] Zhiheng Huang, Wei Xu, and Kai Yu. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991, 2015.","title":"Bidirectional lstm-crf models for sequence tagging","authors":"Zhiheng Huang, Wei Xu, Kai Yu","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jiang et al., 2008] W. Jiang, L. Huang, Q. Liu, and Y. Lu. A cascaded linear model for joint Chinese word segmentation and part-of-speech tagging. In ACL. Citeseer, 2008.","title":"A cascaded linear model for joint Chinese word segmentation and part-of-speech tagging","authors":"W Jiang, L Huang, Q Liu, Y Lu","year":"2008","venue":"ACL. Citeseer","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jiang et al., 2009] W. Jiang, L. Huang, and Q. Liu. Auto- matic adaptation of annotation standards: Chinese word segmentation and POS tagging: a case study. In ACL, pages 522-530, 2009.","title":"Automatic adaptation of annotation standards: Chinese word segmentation and POS tagging: a case study","authors":"W Jiang, L Huang, Q Liu","year":"2009","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jin and Chen, 2008] C. Jin and X. Chen. The fourth interna- tional chinese language processing bakeoff: Chinese word segmentation, named entity recognition and chinese pos tagging. In Sixth SIGHAN Workshop on Chinese Language Processing, page 69, 2008.","title":"The fourth international chinese language processing bakeoff: Chinese word segmentation, named entity recognition and chinese pos tagging","authors":"C Jin, X Chen","year":"2008","venue":"Sixth SIGHAN Workshop on Chinese Language Processing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kim et al., 2015] Yoon Kim, Yacine Jernite, David Sontag, and Alexander M Rush. Character-aware neural language models. arXiv preprint arXiv:1508.06615, 2015.","title":"Character-aware neural language models","authors":"Yoon Kim, Yacine Jernite, David Sontag, Alexander M Rush","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kruengkrai et al., 2009] Canasai Kruengkrai, Kiyotaka Uchimoto, Jun'ichi Kazama, Yiou Wang, Kentaro Tori- sawa, and Hitoshi Isahara. An error-driven word-character hybrid model for joint chinese word segmentation and pos tagging. In ACL, pages 513-521. Association for Computational Linguistics, 2009.","title":"An error-driven word-character hybrid model for joint chinese word segmentation and pos tagging","authors":"Kruengkrai","year":"2009","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Labeau et al., 2015] Matthieu Labeau, Kevin Löser, and Alexandre Allauzen. Non-lexical neural architecture for fine-grained pos tagging. In EMNLP, pages 232-237, Lis- bon, Portugal, September 2015. Association for Computa- tional Linguistics.","title":"Non-lexical neural architecture for fine-grained pos tagging","authors":"Matthieu Labeau, Kevin Löser, Alexandre Allauzen","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lafferty et al., 2001] John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Con- ference on Machine Learning, 2001.","title":"Conditional random fields: Probabilistic models for segmenting and labeling sequence data","authors":"John D Lafferty, Andrew Mccallum, Fernando C N Pereira","year":"2001","venue":"Proceedings of the Eighteenth International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ma and Hovy, 2016] Xuezhe Ma and Eduard Hovy. End- to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv preprint arXiv:1603.01354, 2016.","title":"Xuezhe Ma and Eduard Hovy. Endto-end sequence labeling via bi-directional lstm-cnns-crf","authors":"Hovy Ma","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Kai Chen, Greg Cor- rado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arX- iv:1301.3781, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Mikolov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pei et al., 2014] Wenzhe Pei, Tao Ge, and Chang Baobao. Maxmargin tensor neural network for chinese word seg- mentation. In Proceedings of ACL, 2014.","title":"Maxmargin tensor neural network for chinese word segmentation","authors":"Pei","year":"2014","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Qian and Liu, 2012] Xian Qian and Yang Liu. Joint chinese word segmentation, pos tagging and parsing. In EMNLP, pages 501-511. Association for Computational Linguistic- s, 2012.","title":"Joint chinese word segmentation, pos tagging and parsing","authors":"Xian Qian, Yang Liu","year":"2012","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Qian et al., 2010] Xian Qian, Qi Zhang, Yaqian Zhou, Xu- anjing Huang, and Lide Wu. Joint training and decoding using virtual nodes for cascaded segmentation and tagging tasks. In EMNLP, 2010.","title":"Joint training and decoding using virtual nodes for cascaded segmentation and tagging tasks","authors":"Xian Qian, Qi Zhang, Yaqian Zhou, Xuanjing Huang, Lide Wu","year":"2010","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Qiu et al., 2013] Xipeng Qiu, Jiayi Zhao, and Xuanjing Huang. Joint chinese word segmentation and pos tag- ging on heterogeneous annotated corpora with multiple task learning. In EMNLP, pages 658-668, 2013.","title":"Joint chinese word segmentation and pos tagging on heterogeneous annotated corpora with multiple task learning","authors":"Qiu","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Santos and Zadrozny, 2014] Cicero D Santos and Bianca Zadrozny. Learning character-level representations for part-of-speech tagging. In ICML, pages 1818-1826, 2014.","title":"Learning character-level representations for part-of-speech tagging","authors":"Santos, Zadrozny ; Cicero, D Santos, Bianca Zadrozny","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shen et al., 2014] Mo Shen, Hongxiao Liu, Daisuke Kawa- hara, and Sadao Kurohashi. Chinese morphological anal- ysis with character-level pos tagging. In ACL, pages 253-258, 2014.","title":"Chinese morphological analysis with character-level pos tagging","authors":"Mo Shen, Hongxiao Liu, Daisuke Kawahara, Sadao Kurohashi","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Srivastava et al., 2015] Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber. Highway networks. arXiv preprint arXiv:1505.00387, 2015.","title":"Highway networks","authors":"Rupesh Kumar Srivastava, Klaus Greff, Jürgen Schmidhuber","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sun and Wan, 2012] Weiwei Sun and Xiaojun Wan. Reduc- ing approximation and estimation errors for Chinese lex- ical processing with heterogeneous annotations. In ACL, pages 232-241, 2012.","title":"Reducing approximation and estimation errors for Chinese lexical processing with heterogeneous annotations","authors":"Wan ; Sun, Weiwei Sun, Xiaojun Wan","year":"2012","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sun, 2011] W. Sun. A stacked sub-word model for joint Chi- nese word segmentation and part-of-speech tagging. In A- CL, pages 1385-1394, 2011.","title":"A stacked sub-word model for joint Chinese word segmentation and part-of-speech tagging","authors":"W Sun","year":"2011","venue":"A-CL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Taskar et al., 2005] Ben Taskar, Vassil Chatalbashev, Daphne Koller, and Carlos Guestrin. Learning structured prediction models: A large margin approach. In ICML, 2005.","title":"Learning structured prediction models: A large margin approach","authors":"Ben Taskar, Vassil Chatalbashev, Daphne Koller, Carlos Guestrin","year":"2005","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tsuboi, 2014] Yuta Tsuboi. Neural networks leverage corpus-wide information for part-of-speech tagging. In EMNLP, pages 938-950, Doha, Qatar, October 2014. As- sociation for Computational Linguistics.","title":"Neural networks leverage corpus-wide information for part-of-speech tagging","authors":"Yuta Tsuboi, Tsuboi","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang and Clark, 2008] Yue Zhang and Stephen Clark. Joint word segmentation and pos tagging using a single perceptron. In ACL, pages 888-896, 2008.","title":"Joint word segmentation and pos tagging using a single perceptron","authors":"Yue Zhang, Stephen Clark","year":"2008","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang and Clark, 2010] Yue Zhang and Stephen Clark. A fast decoder for joint word segmentation and POS-tagging using a single discriminative model. In EMNLP, pages 843-852, Stroudsburg, PA, USA, 2010.","title":"A fast decoder for joint word segmentation and POS-tagging using a single discriminative model","authors":"Yue Zhang, Stephen Clark","year":"2010","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zheng et al., 2013] Xiaoqing Zheng, Hanyang Chen, and Tianyu Xu. Deep learning for chinese word segmentation and pos tagging. In EMNLP, pages 647-657, 2013.","title":"Deep learning for chinese word segmentation and pos tagging","authors":"Xiaoqing Zheng, Hanyang Chen, Tianyu Xu","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}