{"type":"ScholarlyArticle","title":"When Does Label Propagation Fail? A View from a Network Generative Model","authors":[{"name":"Yuto Yamaguchi","slug":"yuto-yamaguchi","given":"Yuto","family":"Yamaguchi","affiliation":"†AIST , Japan"},{"name":"Kohei Hayashi","slug":"kohei-hayashi","given":"Kohei","family":"Hayashi","affiliation":"†AIST , Japan"}],"abstract":"What kinds of data does Label Propagation (LP) work best on? Can we justify the solution of LP from a theoretical standpoint? LP is a semi-supervised learning algorithm that is widely used to predict unobserved node labels on a network (e.g., user's gender on an SNS). Despite its importance, its theoretical properties remain mostly unexplored. In this paper, we answer the above questions by interpreting LP from a statistical viewpoint. As our main result, we identify the network generative model behind the discretized version of LP (DLP), and we show that under specific conditions the solution of DLP is equal to the maximum {\\it a posteriori} estimate of that generative model. Our main result reveals the critical limitations of LP. Specifically, we discover that LP would not work best on networks with (1) disassortative node labels, (2) clusters having different edge densities, (3) non-uniform label distributions, or (4) unreliable node labels provided. Our experiments under a variety of settings support our theoretical results.","doi":"10.24963/ijcai.2017/450","url":"https://ijcai.org/proceedings/2017/450","pdf":"https://ijcai.org/proceedings/2017/0450.pdf","markdown":"https://ijcai.org/proceedings/2017/450.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/450","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":450,"pages":{"first":3224,"last":3230},"keywords":["Machine Learning: Relational Learning","Machine Learning: Semi-Supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Backstrom et al., 2010] Lars Backstrom, Eric Sun, and Cameron Marlow. Find me if you can: improving geo- graphical prediction with social and spatial proximity. In WWW, pages 61-70, 2010.","title":"Find me if you can: improving geographical prediction with social and spatial proximity","authors":"Backstrom","year":"2010","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Baluja et al., 2008] Shumeet Baluja, Rohan Seth, D Sivaku- mar, Yushi Jing, Jay Yagnik, Shankar Kumar, Deepak Ravichandran, and Mohamed Aly. Video suggestion and discovery for youtube: taking random walks through the view graph. In WWW, pages 895-904, 2008.","title":"Video suggestion and discovery for youtube: taking random walks through the view graph","authors":"Baluja","year":"2008","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio et al., 2006] Yoshua Bengio, Olivier Delalleau, and Nicolas Le Roux. Label propagation and quadratic crite- rion. Semi-supervised learning, 10, 2006.","title":"Label propagation and quadratic criterion","authors":"Yoshua Bengio, Olivier Delalleau, Nicolas Le Roux","year":"2006","venue":"Semi-supervised learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chang and Blei, 2010] Jonathan Chang and David M Blei. Hierarchical relational models for document networks. The Annals of Applied Statistics, pages 124-150, 2010.","title":"Hierarchical relational models for document networks","authors":"Chang, Blei ;, Jonathan Chang, David M Blei","year":"2010","venue":"The Annals of Applied Statistics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chaudhari et al., 2014] Gaurish Chaudhari, Vashist Avad- hanula, and Sunita Sarawagi. A few good predictions: se- lective node labeling in a social network. In WSDM, pages 353-362, 2014.","title":"Gaurish Chaudhari, Vashist Avadhanula, and Sunita Sarawagi. A few good predictions: selective node labeling in a social network","authors":"Chaudhari","year":"2014","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cho et al., 2016] Yoon-Sik Cho, Greg Ver Steeg, Emilio Ferrara, and Aram Galstyan. Latent space model for multi- modal social data. In WWW, pages 447-458, 2016.","title":"Latent space model for multimodal social data","authors":"Yoon-Sik Cho, Greg Ver Steeg, Emilio Ferrara, Aram Galstyan","year":"2016","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Daudin et al., 2008] J-J Daudin, Franck Picard, and Stéphane Robin. A mixture model for random graphs. Statistics and computing, 18(2):173-183, 2008.","title":"A mixture model for random graphs","authors":"J-J Daudin, Franck Picard, Stéphane Robin","year":"2008","venue":"Statistics and computing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Harville, 1998] David A Harville. Matrix algebra from a statistician's perspective. Technometrics, 40(2):164-164, 1998.","title":"Matrix algebra from a statistician's perspective","authors":"A David, Harville","year":"1998","venue":"Technometrics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jacob et al., 2014] Yann Jacob, Ludovic Denoyer, and Patrick Gallinari. Learning latent representations of nodes for classifying in heterogeneous social networks. In WSDM, pages 373-382, 2014.","title":"Learning latent representations of nodes for classifying in heterogeneous social networks","authors":"Yann Jacob, Ludovic Denoyer, Patrick Gallinari","year":"2014","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Karrer and Newman, 2011] Brian Karrer and Mark EJ New- man. Stochastic blockmodels and community structure in networks. Physical Review E, 83(1):016107, 2011.","title":"Stochastic blockmodels and community structure in networks","authors":"Brian Karrer, Mark Ej Newman","year":"2011","venue":"Physical Review E","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kim and Leskovec, 2012] Myunghwan Kim and Jure Leskovec. Multiplicative attribute graph model of real- world networks. Internet Mathematics, 8(1-2):113-160, 2012.","title":"Multiplicative attribute graph model of realworld networks","authors":"Leskovec ; Kim, Myunghwan Kim, Jure Leskovec","year":"2012","venue":"Internet Mathematics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kyng et al., 2015] Rasmus Kyng, Anup Rao, Sushant Sachdeva, and Daniel A Spielman. Algorithms for lips- chitz learning on graphs. In Proceedings of The 28th Con- ference on Learning Theory, pages 1190-1223, 2015.","title":"Algorithms for lipschitz learning on graphs","authors":"Rasmus Kyng, Anup Rao, Sushant Sachdeva, Daniel A Spielman","year":"2015","venue":"Proceedings of The 28th Conference on Learning Theory","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lancichinetti et al., 2008] Andrea Lancichinetti, Santo For- tunato, and Filippo Radicchi. Benchmark graphs for test- ing community detection algorithms. Physical review E, 78(4):046110, 2008.","title":"Santo Fortunato, and Filippo Radicchi. Benchmark graphs for testing community detection algorithms","authors":"Andrea Lancichinetti","year":"2008","venue":"Physical review E","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mislove et al., 2010] Alan Mislove, Bimal Viswanath, Kr- ishna P Gummadi, and Peter Druschel. You are who you know: inferring user profiles in online social networks. In WSDM, pages 251-260, 2010.","title":"You are who you know: inferring user profiles in online social networks","authors":"Alan Mislove, Bimal Viswanath, Krishna P Gummadi, Peter Druschel","year":"2010","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nallapati et al., 2008] Ramesh M Nallapati, Amr Ahmed, Eric P Xing, and William W Cohen. Joint latent topic models for text and citations. In KDD, pages 542-550, 2008.","title":"Joint latent topic models for text and citations","authors":"Ramesh M Nallapati, Amr Ahmed, Eric P Xing, William W Cohen","year":"2008","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Newman and Clauset, 2015] MEJ Newman and Aaron Clauset. Structure and inference in annotated networks. arXiv preprint arXiv:1507.04001, 2015.","title":"Structure and inference in annotated networks","authors":"Clauset ; Mej Newman, Aaron Newman, Clauset","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pfeiffer III et al., 2014] Joseph J Pfeiffer III, Sebastian Moreno, Timothy La Fond, Jennifer Neville, and Brian Gallagher. Attributed graph models: Modeling network structure with correlated attributes. In WWW, pages 831- 842, 2014.","title":"Attributed graph models: Modeling network structure with correlated attributes","authors":"Joseph J Pfeiffer, Iii, Sebastian Moreno, Timothy La Fond, Jennifer Neville, Brian Gallagher","year":"2014","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Saade et al., 2014] Alaa Saade, Florent Krzakala, and Lenka Zdeborová. Spectral clustering of graphs with the bethe hessian. In NIPS, pages 406-414, 2014.","title":"Spectral clustering of graphs with the bethe hessian","authors":"Alaa Saade, Florent Krzakala, Lenka Zdeborová","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sen et al., 2008] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine, 29(3):93, 2008.","title":"Collective classification in network data","authors":"Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, Tina Eliassi-Rad","year":"2008","venue":"AI magazine","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Takac and Zabovsky, 2012] Lubos Takac and Michal Zabovsky. Data analysis in public social networks. In International Scientific Conference AND International Workshop Present Day Trends of Innovations, 2012.","title":"Data analysis in public social networks","authors":"Zabovsky Takac, Lubos Takac, Michal Zabovsky","year":"2012","venue":"International Scientific Conference AND International Workshop Present Day Trends of Innovations","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Talukdar and Crammer, 2009] Partha Pratim Talukdar and Koby Crammer. New regularized algorithms for transduc- tive learning. In ECMLPKDD, pages 442-457, 2009.","title":"New regularized algorithms for transductive learning","authors":"Partha Pratim, Talukdar, Koby Crammer","year":"2009","venue":"ECMLPKDD","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang and Wong, 1987] Yuchung J Wang and George Y Wong. Stochastic blockmodels for directed graphs. Jour- nal of the American Statistical Association, 82(397):8-19, 1987.","title":"Stochastic blockmodels for directed graphs","authors":"Wang, Wong ; ] Yuchung, J Wang, George Y Wong","year":"1987","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yamaguchi et al., 2015] Yuto Yamaguchi, Christos Falout- sos, and Hiroyuki Kitagawa. Omni-prop: Seamless node classification on arbitrary label correlation. In AAAI, pages 3122-3128, 2015.","title":"Omni-prop: Seamless node classification on arbitrary label correlation","authors":"Yuto Yamaguchi, Christos Faloutsos, Hiroyuki Kitagawa","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2014] Pan Zhang, Cristopher Moore, and Lenka Zdeborová. Phase transitions in semisupervised clustering of sparse networks. Physical Review E, 90(5):052802, 2014.","title":"Phase transitions in semisupervised clustering of sparse networks","authors":"Zhang","year":"2014","venue":"Physical Review E","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2016] Pan Zhang, Cristopher Moore, and MEJ Newman. Community detection in networks with unequal groups. Physical Review E, 93(1):012303, 2016.","title":"Community detection in networks with unequal groups","authors":"Zhang","year":"2016","venue":"Physical Review E","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou et al., 2004] Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Schölkopf. Learning with local and global consistency. pages 321-328, 2004.","title":"Learning with local and global consistency","authors":"Zhou","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhu et al., 2003] Xiaojin Zhu, Zoubin Ghahramani, John Lafferty, et al. Semi-supervised learning using gaussian fields and harmonic functions. In ICML, volume 3, pages 912-919, 2003.","title":"Semi-supervised learning using gaussian fields and harmonic functions","authors":"Xiaojin Zhu, Zoubin Ghahramani, John Lafferty","year":"2003","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhu et al., 2005] Xiaojin Zhu, John Lafferty, and Ronald Rosenfeld. Semi-supervised learning with graphs. Carnegie Mellon University, language technologies insti- tute, school of computer science, 2005.","title":"Semi-supervised learning with graphs","authors":"Xiaojin Zhu, John Lafferty, Ronald Rosenfeld","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}