{"type":"ScholarlyArticle","title":"Socialized Word Embeddings","authors":[{"name":"Ziqian Zeng","slug":"ziqian-zeng","given":"Ziqian","family":"Zeng","affiliation":"Department of CSE , HKUST , Hong Kong ."},{"name":"Yichun Yin","slug":"yichun-yin","given":"Yichun","family":"Yin","affiliation":"School of EECS , Peking University , China.; Department of CSE , HKUST , Hong Kong ."},{"name":"Yangqiu Song","slug":"yangqiu-song","given":"Yangqiu","family":"Song","affiliation":"Department of CSE , HKUST , Hong Kong ."},{"name":"Ming Zhang","slug":"ming-zhang","given":"Ming","family":"Zhang","affiliation":"School of EECS , Peking University , China."}],"abstract":"Word embeddings have attracted a lot of attention. On social media, each user’s language use can be significantly affected by the user’s friends. In this paper, we propose a socialized word embedding algorithm which can consider both user’s personal characteristics of language use and the user’s social relationship on social media. To incorporate personal characteristics, we propose to use a user vector to represent each user. Then for each user, the word embeddings are trained based on each user’s corpus by combining the global word vectors and local user vector. To incorporate social relationship, we add a regularization term to impose similarity between two friends. In this way, we can train the global word vectors and user vectors jointly. To demonstrate the effectiveness, we used the latest large-scale Yelp data to train our vectors, and designed several experiments to show how user vectors affect the results.","doi":"10.24963/ijcai.2017/547","url":"https://ijcai.org/proceedings/2017/547","pdf":"https://ijcai.org/proceedings/2017/0547.pdf","markdown":"https://ijcai.org/proceedings/2017/547.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/547","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":547,"pages":{"first":3915,"last":3921},"keywords":["Natural Language Processing: Sentiment Analysis and Text Mining","Multidisciplinary Topics and Applications: Personalization and User Modeling"],"videos":[],"references":[{"position":1,"raw":"[Bengio et al., 2003] Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. A neural probabilistic language model. JMLR, 3:1137-1155, 2003.","title":"A neural probabilistic language model","authors":"Yoshua Bengio, Réjean Ducharme, Pascal Vincent, Christian Janvin","year":"2003","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Caruana, 1997] Rich Caruana. Multitask learning. Machine Learning, 28(1):41-75, 1997.","title":"Multitask learning","authors":"Caruana","year":"1997","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2016] Huimin Chen, Maosong Sun, Cunchao Tu, Yankai Lin, and Zhiyuan Liu. Neural sentiment classification with user and product attention. In EMNLP, pages 1650-1659, 2016.","title":"Neural sentiment classification with user and product attention","authors":"Huimin Chen, Maosong Sun, Cunchao Tu, Yankai Lin, Zhiyuan Liu","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Collobert et al., 2011] Ronan Collobert, Jason Weston, Léon Bot- tou, Michael Karlen, Koray Kavukcuoglu, and Pavel P. Kuksa. Natural language processing (almost) from scratch. JMLR, 12:2493-2537, 2011.","title":"Natural language processing (almost) from scratch","authors":"Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel P Kuksa","year":"2011","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Croft et al., 2001] W. Bruce Croft, Stephen Cronen-Townsend, and Victor Lavrenko. Relevance feedback and personalization: A language modeling perspective. In DELOS Workshop: Person- alisation and Recommender Systems in Digital Libraries, 2001.","title":"Relevance feedback and personalization: A language modeling perspective","authors":"W Bruce Croft, Stephen Cronen-Townsend, Victor Lavrenko","year":"2001","venue":"DELOS Workshop: Personalisation and Recommender Systems in Digital Libraries","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dubois and Horvath, 1998] Sylvie Dubois and Barbara Horvath. Let's tink about dat: Interdental fricatives in cajun english. Lan- guage Variation and Change, 10(3):245-61, 1998.","title":"Let's tink about dat: Interdental fricatives in cajun english","authors":"Horvath Dubois, Sylvie Dubois, Barbara Horvath","year":"1998","venue":"Language Variation and Change","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2016] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. http: //www.deeplearningbook.org.","title":"Deep Learning","authors":"Ian Goodfellow, Yoshua Bengio, Aaron Courville","year":"2016","venue":null,"doi":null,"url":"http://www.deeplearningbook.org","internal_url":null},{"position":8,"raw":"[Holmes, 2012] Janet Holmes. An introduction to sociolinguistics. Pearson, 4th ed edition, 2012.","title":"An introduction to sociolinguistics","authors":"Janet Holmes","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hu et al., 2013] Xia Hu, Lei Tang, Jiliang Tang, and Huan Liu. Ex- ploiting social relations for sentiment analysis in microblogging. In WSDM, pages 537-546, 2013.","title":"Exploiting social relations for sentiment analysis in microblogging","authors":"Xia Hu, Lei Tang, Jiliang Tang, Huan Liu","year":"2013","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2014] Yu-Yang Huang, Rui Yan, Tsung-Ting Kuo, and Shou-De Lin. Enriching cold start personalized language model using social network information. In ACL, pages 611-617, 2014.","title":"Enriching cold start personalized language model using social network information","authors":"Yu-Yang Huang, Rui Yan, Tsung-Ting Kuo, Shou-De Lin","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huffman, 1952] David A. Huffman. A method for the construc- tion of minimum-redundancy codes. Proceedings of the IRE, 40(9):1098-1101, 1952.","title":"A method for the construction of minimum-redundancy codes","authors":"David A Huffman","year":"1952","venue":"Proceedings of the IRE","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Józefowicz et al., 2016] Rafal Józefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. Exploring the lim- its of language modeling. CoRR, abs/1602.02410, 2016.","title":"Exploring the limits of language modeling","authors":"Rafal Józefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, Yonghui Wu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jurafsky and Martin, 2008] Daniel Jurafsky and James H Martin. Speech and language processing, 2nd edition, 2008.","title":"Speech and language processing","authors":"Daniel Jurafsky, James H Martin","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lazarsfeld and Merton, 1954] Paul F Lazarsfeld and Robert K Merton. Friendship as a social process: A substantive and methodological analysis. Freedom and control in modern society, 18:18-66, 1954.","title":"Friendship as a social process: A substantive and methodological analysis","authors":"F Paul, Robert K Lazarsfeld, Merton","year":"1954","venue":"Freedom and control in modern society","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Levy and Goldberg, 2014] Omer Levy and Yoav Goldberg. Neural word embedding as implicit matrix factorization. In NIPS, pages 2177-2185, 2014.","title":"Neural word embedding as implicit matrix factorization","authors":"Goldberg Levy, Omer Levy, Yoav Goldberg","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Levy et al., 2015] Omer Levy, Yoav Goldberg, and Ido Dagan. Im- proving distributional similarity with lessons learned from word embeddings. TACL, 3:211-225, 2015.","title":"Improving distributional similarity with lessons learned from word embeddings","authors":"Omer Levy, Yoav Goldberg, Ido Dagan","year":"2015","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[McPherson et al., 2001] Miller McPherson, Lynn Smith-Lovin, and James M Cook. Birds of a feather: Homophily. Annu. Rev. Sociol, 27:415-44, 2001.","title":"Birds of a feather: Homophily","authors":"Lynn Miller Mcpherson, James M Smith-Lovin, Cook","year":"2001","venue":"Annu. Rev. Sociol","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mikolov et al., 2013a] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Mikolov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mikolov et al., 2013b] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111-3119. 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mitra and Craswell, 2017] Bhaskar Mitra and Nick Craswell. Neural text embeddings for information retrieval. In WSDM, pages 813-814, 2017.","title":"Neural text embeddings for information retrieval","authors":"Mitra Bhaskar, Nick Craswell","year":"2017","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mnih and Hinton, 2008] Andriy Mnih and Geoffrey E. Hinton. A scalable hierarchical distributed language model. In NIPS, pages 1081-1088, 2008.","title":"A scalable hierarchical distributed language model","authors":"Andriy Mnih, Geoffrey E Hinton","year":"2008","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Morin and Bengio, 2005] Frederic Morin and Yoshua Bengio. Hi- erarchical probabilistic neural network language model. In AIS- TATS, 2005.","title":"Hierarchical probabilistic neural network language model","authors":"Bengio Morin, Frederic Morin, Yoshua Bengio","year":"2005","venue":"AIS-TATS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nguyen et al., 2016] Dong Nguyen, A. Seza Dogruöz, Car- olyn Penstein Rosé, and Franciska de Jong. Computational so- ciolinguistics: A survey. Computational Linguistics, 42(3):537- 593, 2016.","title":"Carolyn Penstein Rosé, and Franciska de Jong. Computational sociolinguistics: A survey","authors":"Nguyen","year":"2016","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word repre- sentation. In EMNLP, volume 14, pages 1532-43, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Song and Lee, 2017] Yan Song and Chia-Jung Lee. Learning user embeddings from emails. In EACL, pages 733-738, 2017.","title":"Learning user embeddings from emails","authors":"Lee ; Song, Yan Song, Chia-Jung Lee","year":"2017","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Song et al., 2010] Wei Song, Yu Zhang, Ting Liu, and Sheng Li. Bridging topic modeling and personalized search. In COLING, pages 1167-1175, 2010.","title":"Bridging topic modeling and personalized search","authors":"Wei Song, Yu Zhang, Ting Liu, Sheng Li","year":"2010","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Song et al., 2013] Yangqiu Song, Zhengdong Lu, Cane Wing-ki Leung, and Qiang Yang. Collaborative boosting for activity clas- sification in microblogs. In KDD, pages 482-490, 2013.","title":"Collaborative boosting for activity classification in microblogs","authors":"Song","year":"2013","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sontag et al., 2012] David Sontag, Kevyn Collins-Thompson, Paul N. Bennett, Ryen W. White, Susan T. Dumais, and Bodo Billerbeck. Probabilistic models for personalizing web search. In WSDM, pages 433-442, 2012.","title":"Probabilistic models for personalizing web search","authors":"David Sontag, Kevyn Collins-Thompson, Paul N Bennett, Ryen W White, Susan T Dumais, Bodo Billerbeck","year":"2012","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sun et al., 2005] Jian-Tao Sun, Hua-Jun Zeng, Huan Liu, Yuchang Lu, and Zheng Chen. Cubesvd: a novel approach to personalized web search. In WWW, pages 382-390, 2005.","title":"Cubesvd: a novel approach to personalized web search","authors":"Jian-Tao Sun, Hua-Jun Zeng, Huan Liu, Yuchang Lu, Zheng Chen","year":"2005","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Tang et al., 2015a] Duyu Tang, Bing Qin, and Ting Liu. Document modeling with gated recurrent neural network for sentiment clas- sification. In EMNLP, pages 1422-1432, 2015.","title":"Document modeling with gated recurrent neural network for sentiment classification","authors":"Tang","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Tang et al., 2015b] Duyu Tang, Bing Qin, and Ting Liu. Learn- ing semantic representations of users and products for document level sentiment classification. In ACL, pages 1014-1023, 2015.","title":"Learning semantic representations of users and products for document level sentiment classification","authors":"Tang","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tang et al., 2015c] Duyu Tang, Bing Qin, Ting Liu, and Yuekui Yang. User modeling with neural network for review rating pre- diction. In IJCAI, pages 1340-1346, 2015.","title":"User modeling with neural network for review rating prediction","authors":"Tang","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/193"},{"position":33,"raw":"[Teevan et al., 2009] Jaime Teevan, Meredith Ringel Morris, and Steve Bush. Discovering and using groups to improve person- alized search. In WSDM, pages 15-24, 2009.","title":"Discovering and using groups to improve personalized search","authors":"Jaime Teevan, Meredith Ringel Morris, Steve Bush","year":"2009","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Turian et al., 2010] Joseph Turian, Lev-Arie Ratinov, and Yoshua Bengio. Word representations: A simple and general method for semi-supervised learning. In ACL, pages 384-394, 2010.","title":"Word representations: A simple and general method for semi-supervised learning","authors":"Joseph Turian, Lev-Arie Ratinov, Yoshua Bengio","year":"2010","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Vosecky et al., 2014] Jan Vosecky, Kenneth Wai-Ting Leung, and Wilfred Ng. Collaborative personalized twitter search with topic- language models. In SIGIR, pages 53-62, 2014.","title":"Collaborative personalized twitter search with topiclanguage models","authors":"Vosecky","year":"2014","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wen et al., 2013] Tsung-Hsien Wen, Aaron Heidel, Hung-yi Lee, Yu Tsao, and Lin-Shan Lee. Recurrent neural network based lan- guage model personalization by social network crowdsourcing. In INTERSPEECH, pages 2703-2707, 2013.","title":"Recurrent neural network based language model personalization by social network crowdsourcing","authors":"Tsung-Hsien Wen, Aaron Heidel, Hung-Yi Lee, Yu Tsao, Lin-Shan Lee","year":"2013","venue":"INTERSPEECH","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Wu and Huang, 2016] Fangzhao Wu and Yongfeng Huang. Per- sonalized microblog sentiment classification via multi-task learn- ing. In AAAI, pages 3059-3065, 2016.","title":"Personalized microblog sentiment classification via multi-task learning","authors":"Huang Wu, Fangzhao Wu, Yongfeng Huang","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Xue et al., 2009] Gui-Rong Xue, Jie Han, Yong Yu, and Qiang Yang. User language model for collaborative personalized search. ACM TOIS, 27(2):11:1-11:28, 2009.","title":"User language model for collaborative personalized search","authors":"Gui-Rong Xue, Jie Han, Yong Yu, Qiang Yang","year":"2009","venue":"ACM TOIS","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Yan et al., 2016] Rui Yan, Cheng-Te Li, Hsun-Ping Hsieh, Po Hu, Xiaohua Hu, and Tingting He. Socialized language model smoothing via bi-directional influence propagation on social net- works. In WWW, pages 1395-1406, 2016.","title":"Socialized language model smoothing via bi-directional influence propagation on social networks","authors":"Rui Yan, Cheng-Te Li, Hsun-Ping Hsieh, Po Hu, Xiaohua Hu, Tingting He","year":"2016","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Yu et al., 2016] Yang Yu, Xiaojun Wan, and Xinjie Zhou. User embedding for scholarly microblog recommendation. In ACL, pages 449-453, 2016.","title":"User embedding for scholarly microblog recommendation","authors":"Yang Yu, Xiaojun Wan, Xinjie Zhou","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}