{"type":"ScholarlyArticle","title":"Biased Random Walk based Social Regularization for Word Embeddings","authors":[{"name":"Ziqian Zeng","slug":"ziqian-zeng","given":"Ziqian","family":"Zeng","affiliation":"Department of CSE , The Hong Kong University of Science and Technology"},{"name":"Xin Liu","slug":"xin-liu","given":"Xin","family":"Liu","affiliation":"Department of CSE , The Hong Kong University of Science and Technology; School of Data and Computer Science , Sun Yat-sen University"},{"name":"Yangqiu Song","slug":"yangqiu-song","given":"Yangqiu","family":"Song","affiliation":"Department of CSE , The Hong Kong University of Science and Technology"}],"abstract":"Nowadays, people publish a lot of natural language texts on social media.\r\nSocialized word embeddings (SWE) has been proposed to deal with two phenomena of language use: everyone has his/her own personal characteristics of language use and socially connected users are likely to use language in similar ways.\r\nWe observe that the spread of language use is transitive. \r\nNamely, one user can affect his/her friends and the friends can also affect their friends. \r\nHowever, SWE modeled the transitivity implicitly.\r\nThe social regularization in SWE only applies to one-hop neighbors and thus users outside the one-hop social circle will not be affected directly. \r\nIn this work, we adopt random walk methods to generate paths on the social graph to model the transitivity explicitly. \r\nEach user on a path will be affected by his/her adjacent user(s) on the path.\r\nMoreover, according to the update mechanism of SWE, fewer friends a user has, fewer update opportunities he/she can get. \r\nHence, we propose a biased random walk method to provide these users with more update opportunities.  \r\nExperiments show that our random walk based social regularizations perform better on sentiment classification.","doi":"10.24963/ijcai.2018/634","url":"https://ijcai.org/proceedings/2018/634","pdf":"https://ijcai.org/proceedings/2018/0634.pdf","markdown":"https://ijcai.org/proceedings/2018/634.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/634","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":634,"pages":{"first":4560,"last":4566},"keywords":["Natural Language Processing: Sentiment Analysis and Text Mining","Humans and AI: Personalization and User Modeling","Natural Language Processing: Embeddings"],"videos":[],"references":[{"position":1,"raw":"[Chen et al., 2016] Huimin Chen, Maosong Sun, Cunchao Tu, Yankai Lin, and Zhiyuan Liu. Neural sentiment classi- fication with user and product attention. In EMNLP, pages 1650-1659, 2016.","title":"Neural sentiment classification with user and product attention","authors":"Huimin Chen, Maosong Sun, Cunchao Tu, Yankai Lin, Zhiyuan Liu","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Croft et al., 2001] W Bruce Croft, Stephen Cronen- Townsend, and Victor Lavrenko. Relevance feedback and personalization: A language modeling perspective. In DELOS Workshop: Personalisation and Recommender Systems in Digital Libraries, volume 3, page 13, 2001.","title":"Relevance feedback and personalization: A language modeling perspective","authors":"Bruce Croft, Stephen Cronen-Townsend, Victor Lavrenko","year":"2001","venue":"DELOS Workshop: Personalisation and Recommender Systems in Digital Libraries","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Eckert and McConnell-Ginet, 2003] Penelope Eckert and Sally McConnell-Ginet. Language and gender. Cam- bridge University Press, 2003.","title":"Penelope Eckert and Sally McConnell-Ginet. Language and gender","authors":"Mcconnell-Ginet Eckert","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fischer, 1958] John L Fischer. Social influences on the choice of a linguistic variant. Word, 14(1):47-56, 1958.","title":"Social influences on the choice of a linguistic variant","authors":"L John, Fischer","year":"1958","venue":"Word","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Green, 2002] Lisa J Green. African American English: a linguistic introduction. Cambridge University Press, 2002.","title":"African American English: a linguistic introduction","authors":"Lisa J Green","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In KDD, pages 855-864, 2016.","title":"Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks","authors":"Leskovec Grover","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Harris, 1954] Zellig S Harris. Distributional structure. Word, 10(2-3):146-162, 1954.","title":"Distributional structure","authors":"S Zellig, Harris","year":"1954","venue":"Word","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hovy, 2015] Dirk Hovy. Demographic factors improve clas- sification performance. In ACL (1), pages 752-762, 2015.","title":"Demographic factors improve classification performance","authors":"Dirk Hovy, Hovy","year":"2015","venue":"ACL (1)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2014] Yu-Yang Huang, Rui Yan, Tsung-Ting Kuo, and Shou-De Lin. Enriching cold start personalized language model using social network information. In ACL, volume 2, pages 611-617, 2014.","title":"Enriching cold start personalized language model using social network information","authors":"Yu-Yang Huang, Rui Yan, Tsung-Ting Kuo, Shou-De Lin","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Labov, 1963] William Labov. The social motivation of a sound change. Word, 19(3):273-309, 1963.","title":"The social motivation of a sound change","authors":"William Labov","year":"1963","venue":"Word","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Mikolov et al., 2013a] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Mikolov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mikolov et al., 2013b] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositionality. In NIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Pan et al., 2004] Jia-Yu Pan, Hyung-Jeong Yang, Christos Faloutsos, and Pinar Duygulu. Automatic multimedia cross-modal correlation discovery. In KDD, pages 653- 658, 2004.","title":"Automatic multimedia cross-modal correlation discovery","authors":"Jia-Yu Pan, Hyung-Jeong Yang, Christos Faloutsos, Pinar Duygulu","year":"2004","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social rep- resentations. In KDD, pages 701-710, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rosenthal and McKeown, 2011] Sara Rosenthal and Kath- leen McKeown. Age prediction in blogs: A study of style, content, and online behavior in pre-and post-social media generations. In ACL, pages 763-772, 2011.","title":"Age prediction in blogs: A study of style, content, and online behavior in pre-and post-social media generations","authors":"Mckeown Rosenthal, Sara Rosenthal, Kathleen Mckeown","year":"2011","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Song et al., 2010] Wei Song, Yu Zhang, Ting Liu, and Sheng Li. Bridging topic modeling and personalized search. In COLING: Posters, pages 1167-1175, 2010.","title":"Bridging topic modeling and personalized search","authors":"Wei Song, Yu Zhang, Ting Liu, Sheng Li","year":"2010","venue":"COLING: Posters","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sontag et al., 2012] David Sontag, Kevyn Collins- Thompson, Paul N Bennett, Ryen W White, Susan Dumais, and Bodo Billerbeck. Probabilistic models for personalizing web search. In WSDM, pages 433-442, 2012.","title":"Probabilistic models for personalizing web search","authors":"David Sontag, Kevyn Collins-Thompson, Paul N Bennett, Ryen W White, Susan Dumais, Bodo Billerbeck","year":"2012","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tang et al., 2015] Duyu Tang, Bing Qin, and Ting Liu. Learning semantic representations of users and products for document level sentiment classification. In ACL, pages 1014-1023, 2015.","title":"Learning semantic representations of users and products for document level sentiment classification","authors":"Tang","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Trudgill, 1974] Peter Trudgill. Linguistic change and diffu- sion: Description and explanation in sociolinguistic dialect geography. Language in society, 3(2):215-246, 1974.","title":"Linguistic change and diffusion: Description and explanation in sociolinguistic dialect geography","authors":"Peter Trudgill, Trudgill","year":"1974","venue":"Language in society","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vosecky et al., 2014] Jan Vosecky, Kenneth Wai-Ting Le- ung, and Wilfred Ng. Collaborative personalized twitter search with topic-language models. In SIGIR, pages 53- 62, 2014.","title":"Collaborative personalized twitter search with topic-language models","authors":"Vosecky","year":"2014","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Weiss, 1983] George H Weiss. Random walks and their ap- plications: Widely used as mathematical models, random walks play an important role in several areas of physics, chemistry, and biology. American Scientist, 71(1):65-71, 1983.","title":"Random walks and their applications: Widely used as mathematical models, random walks play an important role in several areas of physics, chemistry, and biology","authors":"H George, Weiss","year":"1983","venue":"American Scientist","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yan et al., 2016] Rui Yan, Cheng-Te Li, Hsun-Ping Hsieh, Po Hu, Xiaohua Hu, and Tingting He. Socialized language model smoothing via bi-directional influence propagation on social networks. In WWW, pages 1395-1406, 2016.","title":"Socialized language model smoothing via bi-directional influence propagation on social networks","authors":"Rui Yan, Cheng-Te Li, Hsun-Ping Hsieh, Po Hu, Xiaohua Hu, Tingting He","year":"2016","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang and Eisenstein, 2017] Yi Yang and Jacob Eisenstein. Overcoming language variation in sentiment analysis with social attention. TACL, 5:295-307, 2017.","title":"Overcoming language variation in sentiment analysis with social attention","authors":"Yang, Eisenstein ; ] Yi, Yang, Jacob Eisenstein","year":"2017","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zeng et al., 2017] Ziqian Zeng, Yichun Yin, Yangqiu Song, and Ming Zhang. Socialized word embeddings. In IJCAI, pages 3915-3921, 2017.","title":"Socialized word embeddings","authors":"Zeng","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/547"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}