{"type":"ScholarlyArticle","title":"Joint POS Tagging and Text Normalization for Informal Text","authors":[{"name":"Chen Li","slug":"chen-li-2","given":"Chen","family":"Li","affiliation":""},{"name":"Yang Liu","slug":"yang-liu-4","given":"Yang","family":"Liu","affiliation":""}],"abstract":"Text normalization and part-of-speech (POS) tagging for social media data have been investigated recently, however, prior work has treated them separately. In this paper, we propose a joint Viterbi decoding process to determine each token’s POS tag and non-standard token’s correct form at the same time. In order to evaluate our approach, we create two new data sets with POS tag labels and non-standard tokens' correct forms. This is the first data set with such annotation. The experiment results demonstrate the effect of non-standard words on POS tagging, and also show that our proposed methods perform better than the state-of-the-art systems in both POS tagging and normalization.","doi":null,"url":"https://ijcai.org/proceedings/2015/182","pdf":"https://ijcai.org/proceedings/2015/0182.pdf","markdown":"https://ijcai.org/proceedings/2015/182.md","bibtex":"https://ijcai.org/proceedings/2015/bibtex/182","volume":{"slug":"2015","title":"Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2015","url":"https://ijcai.org/proceedings/2015/","year":2015,"location":"Buenos Aires, Argentina","dates":"25–31 July 2015","editors":"Qiang Yang and Michael Wooldridge","isbn":"978-1-57735-738-4","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Natural Language Processing","track":"Main Track","number":182,"pages":{"first":1263,"last":1268},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}