{"type":"ScholarlyArticle","title":"Neural Networks Incorporating Unlabeled and Partially-labeled Data for Cross-domain Chinese Word Segmentation","authors":[{"name":"Lujun Zhao","slug":"lujun-zhao","given":"Lujun","family":"Zhao","affiliation":"School of Computer Science , Fudan University , Shanghai , China Shanghai Key Laboratory of Intelligent Information Processing , Shanghai , China"},{"name":"Qi Zhang","slug":"qi-zhang","given":"Qi","family":"Zhang","affiliation":"School of Computer Science , Fudan University , Shanghai , China Shanghai Key Laboratory of Intelligent Information Processing , Shanghai , China"},{"name":"Peng Wang","slug":"peng-wang","given":"Peng","family":"Wang","affiliation":"School of Computer Science , Fudan University , Shanghai , China Shanghai Key Laboratory of Intelligent Information Processing , Shanghai , China"},{"name":"Xiaoyu Liu","slug":"xiaoyu-liu","given":"Xiaoyu","family":"Liu","affiliation":"School of Computer Science , Fudan University , Shanghai , China Shanghai Key Laboratory of Intelligent Information Processing , Shanghai , China"}],"abstract":"Most existing Chinese word segmentation (CWS) methods are usually supervised. Hence, large-scale annotated domain-specific datasets are needed for training. In this paper, we seek to address the problem of CWS for the resource-poor domains that lack annotated data. A novel neural network model is proposed to incorporate unlabeled and partially-labeled data. To make use of unlabeled data, we combine a bidirectional LSTM segmentation model with two character-level language models using a gate mechanism. These language models can capture co-occurrence information. To make use of partially-labeled data, we modify the original cross entropy loss function of RNN. Experimental results demonstrate that the method performs well on CWS tasks in a series of domains.","doi":"10.24963/ijcai.2018/640","url":"https://ijcai.org/proceedings/2018/640","pdf":"https://ijcai.org/proceedings/2018/0640.pdf","markdown":"https://ijcai.org/proceedings/2018/640.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/640","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":640,"pages":{"first":4602,"last":4608},"keywords":["Natural Language Processing: Phonology, Morphology, and word segmentation","Machine Learning: Neural Networks","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Cai and Zhao, 2016] Deng Cai and Hai Zhao. Neural word segmentation learning for chinese. In ACL, 2016.","title":"Neural word segmentation learning for chinese","authors":"Zhao ; ] Deng Cai, Hai Cai, Zhao","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai et al., 2017] Deng Cai, Hai Zhao, Zhisong Zhang, Yuan Xin, Yongjian Wu, and Feiyue Huang. Fast and accu- rate neural word segmentation for chinese. arXiv preprint arXiv:1704.07047, 2017.","title":"Fast and accurate neural word segmentation for chinese","authors":"Deng Cai, Hai Zhao, Zhisong Zhang, Yuan Xin, Yongjian Wu, Feiyue Huang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2015] Xinchi Chen, Xipeng Qiu, Chenxi Zhu, Pengfei Liu, and Xuanjing Huang. Long short-term mem- ory neural networks for chinese word segmentation. In EMNLP, 2015.","title":"Long short-term memory neural networks for chinese word segmentation","authors":"Chen","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the proper- ties of neural machine translation: Encoder-decoder ap- proaches. arXiv preprint arXiv:1409.1259, 2014.","title":"On the properties of neural machine translation: Encoder-decoder approaches","authors":"Cho","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Huang et al., 2007] Chu-Ren Huang, Petr Šimon, Shu-Kai Hsieh, and Laurent Prévot. Rethinking chinese word seg- mentation: tokenization, character classification, or word- break identification. In ACL, 2007.","title":"Rethinking chinese word segmentation: tokenization, character classification, or wordbreak identification","authors":"Chu-Ren Huang, Petr Šimon, Shu-Kai Hsieh, Laurent Prévot","year":"2007","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang et al., 2015] Zhiheng Huang, Wei Xu, and Kai Yu. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991, 2015.","title":"Bidirectional lstm-crf models for sequence tagging","authors":"Zhiheng Huang, Wei Xu, Kai Yu","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2017] Shen Huang, Xu Sun, and Houfeng Wang. Addressing domain adaptation for chinese word segmentation with global recurrent structure. In IJCNLP, 2017.","title":"Addressing domain adaptation for chinese word segmentation with global recurrent structure","authors":"Shen Huang, Xu Sun, Houfeng Wang","year":"2017","venue":"IJCNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jiang et al., 2013] Wenbin Jiang, Meng Sun, Yajuan Lü, Yating Yang, and Qun Liu. Discriminative learning with natural annotations: Word segmentation as a case study. In ACL, 2013.","title":"Discriminative learning with natural annotations: Word segmentation as a case study","authors":"Wenbin Jiang, Meng Sun, Yajuan Lü, Yating Yang, Qun Liu","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jin and Tanaka-Ishii, 2006] Zhihui Jin and Kumiko Tanaka- Ishii. Unsupervised segmentation of chinese text by use of branching entropy. In COLING-ACL, 2006.","title":"Unsupervised segmentation of chinese text by use of branching entropy","authors":"Jin, Tanaka-Ishii ; ] Zhihui, Jin, Kumiko Tanaka-Ishii","year":"2006","venue":"COLING-ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"Diederik Kingma, Jimmy Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li and Sun, 2009] Zhongguo Li and Maosong Sun. Punctu- ation as implicit annotations for chinese word segmenta- tion. Comput. Linguist., 2009.","title":"Punctuation as implicit annotations for chinese word segmentation","authors":"Sun ; Li, Zhongguo Li, Maosong Sun","year":"2009","venue":"Comput. Linguist","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu and Zhang, 2012] Yang Liu and Yue Zhang. Unsuper- vised domain adaptation for joint segmentation and pos- tagging. In COLING, 2012.","title":"Unsupervised domain adaptation for joint segmentation and postagging","authors":"Yang Liu, Yue Zhang","year":"2012","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2014] Yijia Liu, Yue Zhang, Wanxiang Che, Ting Liu, and Fan Wu. Domain adaptation for crf-based chinese word segmentation using free annotations. In EMNLP, 2014.","title":"Domain adaptation for crf-based chinese word segmentation using free annotations","authors":"Yijia Liu, Yue Zhang, Wanxiang Che, Ting Liu, Fan Wu","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2016] Yijia Liu, Wanxiang Che, Jiang Guo, Bing Qin, and Ting Liu. Exploring segment representations for neural segmentation models. In IJCAI, 2016.","title":"Exploring segment representations for neural segmentation models","authors":"Yijia Liu, Wanxiang Che, Jiang Guo, Bing Qin, Ting Liu","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/409"},{"position":16,"raw":"[Peters et al., 2017] Matthew Peters, Waleed Ammar, Chan- dra Bhagavatula, and Russell Power. Semi-supervised se- quence tagging with bidirectional language models. In ACL, 2017.","title":"Semi-supervised sequence tagging with bidirectional language models","authors":"Matthew Peters, Waleed Ammar, Chandra Bhagavatula, Russell Power","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rei, 2017] Marek Rei. Semi-supervised multitask learning for sequence labeling. In ACL, 2017.","title":"Semi-supervised multitask learning for sequence labeling","authors":"Marek Rei, Rei","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shen et al., 2013] Mo Shen, Daisuke Kawahara, and Sadao Kurohashi. Chinese word segmentation by mining maxi- mized substrings. In IJCNLP, 2013.","title":"Chinese word segmentation by mining maximized substrings","authors":"Mo Shen, Daisuke Kawahara, Sadao Kurohashi","year":"2013","venue":"IJCNLP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sun and Xu, 2011] Weiwei Sun and Jia Xu. Enhancing chi- nese word segmentation using unlabeled data. In EMNLP, 2011.","title":"Enhancing chinese word segmentation using unlabeled data","authors":"Xu Sun, Weiwei Sun, Jia Xu","year":"2011","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Triggs and Verbeek, 2008] Bill Triggs and Jakob J Verbeek. Scene segmentation with crfs learned from partially la- beled images. In NIPS, 2008.","title":"Scene segmentation with crfs learned from partially labeled images","authors":"Verbeek ; ] Bill Triggs, Jakob J Triggs, Verbeek","year":"2008","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tsuboi et al., 2008] Yuta Tsuboi, Hisashi Kashima, Hiroki Oda, Shinsuke Mori, and Yuji Matsumoto. Training con- ditional random fields using incomplete annotations. In COLING, 2008.","title":"Training conditional random fields using incomplete annotations","authors":"Yuta Tsuboi, Hisashi Kashima, Hiroki Oda, Shinsuke Mori, Yuji Matsumoto","year":"2008","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xue, 2003] Nianwen Xue. Chinese word segmentation as character tagging. Computational Linguistics and Chinese Language Processing, 2003.","title":"Nianwen Xue. Chinese word segmentation as character tagging","authors":"Xue","year":"2003","venue":"Computational Linguistics and Chinese Language Processing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang and Vozila, 2014] Fan Yang and Paul Vozila. Semi- supervised chinese word segmentation using partial-label learning with conditional random fields. In EMNLP, 2014.","title":"Semisupervised chinese word segmentation using partial-label learning with conditional random fields","authors":"Fan Yang, Paul Vozila","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2017] Jie Yang, Yue Zhang, and Fei Dong. Neu- ral word segmentation with rich pretraining. arXiv preprint arXiv:1704.08960, 2017.","title":"Neural word segmentation with rich pretraining","authors":"Jie Yang, Yue Zhang, Fei Dong","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2013] Longkai Zhang, Li Li, Zhengyan He, Houfeng Wang, and Ni Sun. Improving chinese word seg- mentation on micro-blog using rich punctuations. In ACL, 2013.","title":"Improving chinese word segmentation on micro-blog using rich punctuations","authors":"Zhang","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2014] Meishan Zhang, Yue Zhang, Wanxiang Che, and Ting Liu. Type-supervised domain adaptation for joint segmentation and pos-tagging. In EACL, 2014.","title":"Type-supervised domain adaptation for joint segmentation and pos-tagging","authors":"Zhang","year":"2014","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2016] Meishan Zhang, Yue Zhang, and Guo- hong Fu. Transition-based neural word segmentation. In ACL, 2016.","title":"Transition-based neural word segmentation","authors":"Zhang","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhao and Kit, 2008] Hai Zhao and Chunyu Kit. An empir- ical comparison of goodness measures for unsupervised chinese word segmentation with a unified framework. In IJCNLP, 2008.","title":"An empirical comparison of goodness measures for unsupervised chinese word segmentation with a unified framework","authors":"Kit ; Zhao, Hai Zhao, Chunyu Kit","year":"2008","venue":"IJCNLP","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou et al., 2017] Hao Zhou, Zhenting Yu, Yue Zhang, Shujian Huang, XIN-YU DAI, and Jiajun Chen. Word- context character embeddings for chinese word segmenta- tion. In EMNLP, 2017.","title":"Wordcontext character embeddings for chinese word segmentation","authors":"Hao Zhou, Zhenting Yu, Yue Zhang, Shujian Huang, Xin-Yu Dai, Jiajun Chen","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}