{"type":"ScholarlyArticle","title":"Dataless Short Text Classification Based on Biterm Topic Model and Word Embeddings","authors":[{"name":"Yi Yang","slug":"yi-yang","given":"Yi","family":"Yang","affiliation":"SKLCS , Institute of Software , Chinese Academy of Sciences; University of Chinese Academy of Sciences"},{"name":"Hongan Wang","slug":"hongan-wang","given":"Hongan","family":"Wang","affiliation":"SKLCS , Institute of Software , Chinese Academy of Sciences; Zhejiang Lab; University of Chinese Academy of Sciences"},{"name":"Jiaqi Zhu","slug":"jiaqi-zhu","given":"Jiaqi","family":"Zhu","affiliation":"SKLCS , Institute of Software , Chinese Academy of Sciences; Zhejiang Lab; University of Chinese Academy of Sciences"},{"name":"Yunkun Wu","slug":"yunkun-wu","given":"Yunkun","family":"Wu","affiliation":"University of Chinese Academy of Sciences"},{"name":"Kailong Jiang","slug":"kailong-jiang","given":"Kailong","family":"Jiang","affiliation":"University of Chinese Academy of Sciences"},{"name":"Wenli Guo","slug":"wenli-guo","given":"Wenli","family":"Guo","affiliation":"University of Chinese Academy of Sciences"},{"name":"Wandong Shi","slug":"wandong-shi","given":"Wandong","family":"Shi","affiliation":"University of Chinese Academy of Sciences"}],"abstract":"Dataless text classification has attracted increasing attentions recently. It only needs very few seed words of each category to classify documents, which is much cheaper than supervised text classification that requires massive labeling efforts. However, most of existing models pay attention to long texts, but get unsatisfactory performance on short texts, which have become increasingly popular on the Internet. In this paper, we at first propose a novel model named Seeded Biterm Topic Model (SeedBTM) extending BTM to solve the problem of dataless short text classification with seed words. It takes advantage of both word co-occurrence information in the topic model and category-word similarity from widely used word embeddings as the prior topic-in-set knowledge. Moreover, with the same approach, we also propose Seeded Twitter Biterm Topic Model (SeedTBTM), which extends Twitter-BTM and utilizes additional user information to achieve higher classification accuracy. Experimental results on five real short-text datasets show that our models outperform the state-of-the-art methods, and especially perform well when the categories are overlapping and interrelated.","doi":"10.24963/ijcai.2020/549","url":"https://ijcai.org/proceedings/2020/549","pdf":"https://ijcai.org/proceedings/2020/0549.pdf","markdown":"https://ijcai.org/proceedings/2020/549.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/549","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":549,"pages":{"first":3969,"last":3975},"keywords":["Machine Learning: Knowledge-based Learning","Machine Learning: Learning Graphical Models","Natural Language Processing: Text Classification","Data Mining: Mining Text, Web, Social Media"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25370/file_25370.m3u8","length_sec":283,"provider":"hls"}],"references":[{"position":1,"raw":"[Andrzejewski and Zhu, 2009] David Andrzejewski and Xi- aojin Zhu. Latent Dirichlet allocation with topic-in-set knowledge. In NAACL HLT Workshop on Semi-Supervised Learning for Natural Language Processing, pages 43-48. Association for Computational Linguistics, 2009.","title":"Latent Dirichlet allocation with topic-in-set knowledge","authors":"Zhu Andrzejewski, David Andrzejewski, Xiaojin Zhu","year":"2009","venue":"NAACL HLT Workshop on Semi-Supervised Learning for Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blei et al., 2003] David M Blei, Andrew Y Ng, and Michael I Jordan. Latent Dirichlet allocation. Journal of machine Learning research, 3(Jan):993-1022, 2003.","title":"Latent Dirichlet allocation","authors":"Andrew Y David M Blei, Michael I Ng, Jordan","year":"2003","venue":"Journal of machine Learning research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chang et al., 2008] Ming-Wei Chang, Lev-Arie Ratinov, Dan Roth, and Vivek Srikumar. Importance of semantic representation: Dataless classification. In AAAI, volume 2, pages 830-835, 2008.","title":"Importance of semantic representation: Dataless classification","authors":"Ming-Wei Chang, Lev-Arie Ratinov, Dan Roth, Vivek Srikumar","year":"2008","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2015a] Weizheng Chen, Jinpeng Wang, Yan Zhang, Hongfei Yan, and Xiaoming Li. User based ag- gregation for biterm topic model. In ACL, volume 2 (Short Papers), pages 489-494, 2015.","title":"User based aggregation for biterm topic model","authors":"Chen","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2015b] Xingyuan Chen, Yunqing Xia, Peng Jin, and John Carroll. Dataless text classification with descrip- tive LDA. In AAAI, pages 2224-2231, 2015.","title":"Dataless text classification with descriptive LDA","authors":"Chen","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Druck et al., 2008] Gregory Druck, Gideon Mann, and An- drew McCallum. Learning from labeled features using generalized expectation criteria. In SIGIR, pages 595-602. ACM, 2008.","title":"Learning from labeled features using generalized expectation criteria","authors":"Gregory Druck, Gideon Mann, Andrew Mccallum","year":"2008","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hingmire and Chakraborti, 2014] Swapnil Hingmire and Sutanu Chakraborti. Topic labeled text classification: a weakly supervised approach. In SIGIR, pages 385-394. ACM, 2014.","title":"Swapnil Hingmire and Sutanu Chakraborti. Topic labeled text classification: a weakly supervised approach","authors":"Chakraborti Hingmire","year":"2014","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jiang et al., 2016] Lan Jiang, Hengyang Lu, Ming Xu, and Chongjun Wang. Biterm pseudo document topic model for short text. In ICTAI, pages 865-872. IEEE, 2016.","title":"Biterm pseudo document topic model for short text","authors":"Lan Jiang, Hengyang Lu, Ming Xu, Chongjun Wang","year":"2016","venue":"ICTAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2016a] Chenliang Li, Haoran Wang, Zhiqian Zhang, Aixin Sun, and Zongyang Ma. Topic modeling for short texts with auxiliary word embeddings. In SIGIR, pages 165-174. ACM, 2016.","title":"Topic modeling for short texts with auxiliary word embeddings","authors":"Li","year":"2016","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2016b] Chenliang Li, Jian Xing, Aixin Sun, and Zongyang Ma. Effective document labeling with very few seed words: A topic model approach. In CIKM, pages 85- 94. ACM, 2016.","title":"Effective document labeling with very few seed words: A topic model approach","authors":"Li","year":"2016","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2018a] Chenliang Li, Shiqian Chen, Jian Xing, Aixin Sun, and Zongyang Ma. Seed-guided topic model for document filtering and classification. ACM Transac- tions on Information Systems (TOIS), 37(1):9:1-37, 2018.","title":"Seed-guided topic model for document filtering and classification","authors":"Li","year":"2018","venue":"ACM Transactions on Information Systems (TOIS)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2018b] Ximing Li, Ang Zhang, Changchun Li, Lantian Guo, Wenting Wang, and Jihong Ouyang. Rela- tional biterm topic model: Short-text topic modeling using word embeddings. The Computer Journal, 62(3):359-372, 2018.","title":"Relational biterm topic model: Short-text topic modeling using word embeddings","authors":"Li","year":"2018","venue":"The Computer Journal","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2019] Chenliang Li, Shiqian Chen, and Yan Qi. Filtering and classifying relevant short text with a few seed words. Data and Information Management, 3(3):165-186, 2019.","title":"Filtering and classifying relevant short text with a few seed words","authors":"Li","year":"2019","venue":"Data and Information Management","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2004] Bing Liu, Wee Sun Li, Xiaoli and, and Philip S Yu. Text classification by labeling words. In AAAI, volume 4, pages 425-430, 2004.","title":"Text classification by labeling words","authors":"Bing Liu, Wee, Sun Li, Xiaoli, Philip S Yu","year":"2004","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Meng et al., 2018] Yu Meng, Jiaming Shen, Chao Zhang, and Jiawei Han. Weakly-supervised neural text classifi- cation. In CIKM, pages 983-992. ACM, 2018.","title":"Weakly-supervised neural text classification","authors":"Yu Meng, Jiaming Shen, Chao Zhang, Jiawei Han","year":"2018","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Misra, 2018] Rishabh Misra. News category dataset, June 2018.","title":"News category dataset","authors":"Rishabh Misra","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543. Association for Computational Linguistics, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Quan et al., 2015] Xiaojun Quan, Chunyu Kit, Yong Ge, and Sinno Jialin Pan. Short and sparse text topic modeling via self-aggregation. In IJCAI, pages 2270-2276, 2015.","title":"Short and sparse text topic modeling via self-aggregation","authors":"Xiaojun Quan, Chunyu Kit, Yong Ge, Sinno Jialin Pan","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/321"},{"position":19,"raw":"[Shalaby and Zadrozny, 2019] Walid Shalaby and Wlodek Zadrozny. Learning concept embeddings for data- less classification via efficient bag-of-concepts densifica- tion. Knowledge and Information Systems, 61:1047-1070, 2019.","title":"Learning concept embeddings for dataless classification via efficient bag-of-concepts densification","authors":"Zadrozny ; ] Walid Shalaby, Wlodek Shalaby, Zadrozny","year":"2019","venue":"Knowledge and Information Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Türker, 2019] Rima Türker. Knowledge-based dataless text categorization. In ESWC, pages 231-241. Springer, 2019.","title":"Knowledge-based dataless text categorization","authors":"Rima Türker","year":"2019","venue":"ESWC","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2017] Jin Wang, Zhongyuan Wang, Dawei Zhang, and Jun Yan. Combining knowledge with deep convolutional neural networks for short text classification. In IJCAI, pages 2915-2921, 2017.","title":"Combining knowledge with deep convolutional neural networks for short text classification","authors":"Jin Wang, Zhongyuan Wang, Dawei Zhang, Jun Yan","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/406"},{"position":22,"raw":"[Weng et al., 2010] Jianshu Weng, Ee-Peng Lim, Jing Jiang, and Qi He. Twitterrank: finding topic-sensitive influential Twitterers. In WSDM, pages 261-270. ACM, 2010.","title":"Twitterrank: finding topic-sensitive influential Twitterers","authors":"Weng","year":"2010","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xun et al., 2016] Guangxu Xun, Vishrawas Gopalakrish- nan, Fenglong Ma, Yaliang Li, Jing Gao, and Aidong Zhang. Topic discovery for short texts using word em- beddings. In ICDM, pages 1299-1304. IEEE, 2016.","title":"Topic discovery for short texts using word embeddings","authors":"Guangxu Xun, Fenglong Vishrawas Gopalakrishnan, Yaliang Ma, Jing Li, Aidong Gao, Zhang","year":"2016","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yan et al., 2013] Xiaohui Yan, Jiafeng Guo, Yanyan Lan, and Xueqi Cheng. A biterm topic model for short texts. In WWW, pages 1445-1456. ACM, 2013.","title":"A biterm topic model for short texts","authors":"Xiaohui Yan, Jiafeng Guo, Yanyan Lan, Xueqi Cheng","year":"2013","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yin and Wang, 2014] Jianhua Yin and Jianyong Wang. A Dirichlet multinomial mixture model-based approach for short text clustering. In SIGKDD, pages 233-242. ACM, 2014.","title":"A Dirichlet multinomial mixture model-based approach for short text clustering","authors":"Jianhua Yin, Jianyong Wang","year":"2014","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zeng et al., 2018] Jichuan Zeng, Jing Li, Yan Song, Cuiyun Gao, Michael R Lyu, and Irwin King. Topic memory networks for short text classification. In EMNLP, pages 3120-3131. Association for Computational Linguistics, 2018.","title":"Topic memory networks for short text classification","authors":"Jichuan Zeng, Jing Li, Yan Song, Cuiyun Gao, Irwin Michael R Lyu, King","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2015] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In NIPS, pages 649-657, 2015.","title":"Character-level convolutional networks for text classification","authors":"Xiang Zhang, Junbo Zhao, Yann Lecun","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhao et al., 2011] Wayne Xin Zhao, Jing Jiang, Jianshu Weng, Jing He, Ee-Peng Lim, Hongfei Yan, and Xiaoming Li. Comparing Twitter and traditional media using topic models. In ECIR, pages 338-349. Springer, 2011.","title":"Comparing Twitter and traditional media using topic models","authors":"Jing Wayne Xin Zhao, Jianshu Jiang, Jing Weng, Ee-Peng He, Hongfei Lim, Xiaoming Yan, Li","year":"2011","venue":"ECIR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zuo et al., 2016] Yuan Zuo, Jichang Zhao, and Ke Xu. Word network topic model: a simple but general solution for short and imbalanced texts. Knowledge and Informa- tion Systems, 48(2):379-398, 2016.","title":"Word network topic model: a simple but general solution for short and imbalanced texts","authors":"Yuan Zuo, Jichang Zhao, Ke Xu","year":"2016","venue":"Knowledge and Information Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}