{"type":"ScholarlyArticle","title":"Small-Variance Asymptotics for Nonparametric Bayesian Overlapping Stochastic Blockmodels","authors":[{"name":"Gundeep Arora","slug":"gundeep-arora","given":"Gundeep","family":"Arora","affiliation":"Indian Institute of Technology , Kanpur"},{"name":"Anupreet Porwal","slug":"anupreet-porwal","given":"Anupreet","family":"Porwal","affiliation":"Indian Institute of Technology , Kanpur"},{"name":"Kanupriya Agarwal","slug":"kanupriya-agarwal","given":"Kanupriya","family":"Agarwal","affiliation":"Indian Institute of Technology , Kanpur"},{"name":"Avani Samdariya","slug":"avani-samdariya","given":"Avani","family":"Samdariya","affiliation":"Indian Institute of Technology , Kanpur"},{"name":"Piyush Rai","slug":"piyush-rai","given":"Piyush","family":"Rai","affiliation":"Indian Institute of Technology , Kanpur"}],"abstract":"The latent feature relational model (LFRM) for graphs represents each node as having binary memberships in one or more communities. The community memberships can be represented in form of a binary vector and LFRM defines the link probability between any pair of nodes as a bilinear function of their community membership vectors.  Moreover, using nonparametric Bayesian prior - Indian Buffet Process - on the community membership matrix enables learning the number of communities automatically from the data. However, despite its modeling flexibility, strong link predictive performance, and nice interpretability of binary embeddings, inference in LFRM remains a challenge and is typically done via MCMC or variational methods. These methods can be slow and may take a long time to converge. In this work, we apply the small variance asymptotics idea to the non-parametric Bayesian LFRM, utilizing the connection between exponential families and Bregman divergence. This leads to an overlapping k-means like objective function for the nonparametric Bayesian LFRM, which can be optimized using generic or specialized solvers. We also propose an iterative greedy algorithm to optimize the objective function and compare our approach with other inference methods on several benchmark datasets. Our results demonstrate that our inference algorithm is competitive to methods such as MCMC while being much faster.","doi":"10.24963/ijcai.2018/276","url":"https://ijcai.org/proceedings/2018/276","pdf":"https://ijcai.org/proceedings/2018/0276.pdf","markdown":"https://ijcai.org/proceedings/2018/276.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/276","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":276,"pages":{"first":2000,"last":2006},"keywords":["Machine Learning: Probabilistic Machine Learning","Machine Learning: Relational Learning"],"videos":[],"references":[{"position":1,"raw":"[Airoldi et al., 2008] Edoardo M Airoldi, David M Blei, Stephen E Fienberg, and Eric P Xing. Mixed membership stochastic blockmodels. Journal of Machine Learning Re- search, 9(Sep):1981-2014, 2008.","title":"Mixed membership stochastic blockmodels","authors":"David M Edoardo M Airoldi, Stephen E Blei, Eric P Fienberg, Xing","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Banerjee et al., 2005] Arindam Banerjee, Srujana Merugu, Inderjit S Dhillon, and Joydeep Ghosh. Clustering with bregman divergences. Journal of machine learning re- search, 6(Oct):1705-1749, 2005.","title":"Clustering with bregman divergences","authors":"Arindam Banerjee, Srujana Merugu, Joydeep Inderjit S Dhillon, Ghosh","year":"2005","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Broderick et al., 2013] Tamara Broderick, Brian Kulis, and Michael I Jordan. Mad-bayes: Map-based asymptotic derivations from bayes. In ICML (3), pages 226-234, 2013.","title":"Mad-bayes: Map-based asymptotic derivations from bayes","authors":"Tamara Broderick, Brian Kulis, Michael I Jordan","year":"2013","venue":"ICML (3)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Butland et al., 2005] Gareth Butland, José Manuel Peregrín-Alvarez, Joyce Li, Wehong Yang, Xiaochun Yang, Veronica Canadien, Andrei Starostine, Dawn Richards, Bryan Beattie, Nevan Krogan, et al. Interaction network containing conserved and essential protein complexes in escherichia coli. Nature, 433(7025):531, 2005.","title":"Interaction network containing conserved and essential protein complexes in escherichia coli","authors":"Gareth Butland, José Manuel Peregrín-Alvarez, Joyce Li, Wehong Yang, Xiaochun Yang, Veronica Canadien, Andrei Starostine, Dawn Richards, Bryan Beattie, Nevan Krogan","year":"2005","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gopalan et al., 2012] Prem K Gopalan, Sean Gerrish, Michael Freedman, David M Blei, and David M Mimno. Scalable inference of overlapping communities. In Ad- vances in Neural Information Processing Systems, pages 2249-2257, 2012.","title":"Scalable inference of overlapping communities","authors":"Sean Prem K Gopalan, Michael Gerrish, David M Freedman, David M Blei, Mimno","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Griffiths and Ghahramani, 2011] Thomas L Griffiths and Zoubin Ghahramani. The indian buffet process: An in- troduction and review. Journal of Machine Learning Re- search, 12(Apr):1185-1224, 2011.","title":"The indian buffet process: An introduction and review","authors":"L Thomas, Zoubin Griffiths, Ghahramani","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jiang et al., 2012] Ke Jiang, Brian Kulis, and Michael I Jor- dan. Small-variance asymptotics for exponential family dirichlet process mixture models. In Advances in Neural Information Processing Systems, pages 3158-3166, 2012.","title":"Small-variance asymptotics for exponential family dirichlet process mixture models","authors":"Ke Jiang, Brian Kulis, Michael I Jordan","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jiang et al., 2017] Ke Jiang, Suvrit Sra, and Brian Kulis. Combinatorial topic models using small-variance asymp- totics. In AISTATS, 2017.","title":"Combinatorial topic models using small-variance asymptotics","authors":"Ke Jiang, Suvrit Sra, Brian Kulis","year":"2017","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kemp et al., 2006] Charles Kemp, Joshua B Tenenbaum, Thomas L Griffiths, Takeshi Yamada, and Naonori Ueda. Learning systems of concepts with an infinite relational model. In AAAI, volume 3, page 5, 2006.","title":"Learning systems of concepts with an infinite relational model","authors":"Charles Kemp, Joshua B Tenenbaum, Thomas L Griffiths, Takeshi Yamada, Naonori Ueda","year":"2006","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kulis and Jordan, 2011] Brian Kulis and Michael I Jordan. Revisiting k-means: New algorithms via bayesian non- parametrics. arXiv preprint arXiv:1111.0352, 2011.","title":"Revisiting k-means: New algorithms via bayesian nonparametrics","authors":"Brian Kulis, Michael I Jordan","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lazega, 2001] Emmanuel Lazega. The collegial phe- nomenon: The social mechanisms of cooperation among peers in a corporate law partnership. Oxford University Press on Demand, 2001.","title":"The collegial phenomenon: The social mechanisms of cooperation among peers in a corporate law partnership","authors":"Emmanuel Lazega","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2016] Wenzhe Li, Sungjin Ahn, and Max Welling. Scalable mcmc for mixed membership stochastic block- models. In Artificial Intelligence and Statistics, pages 723-731, 2016.","title":"Scalable mcmc for mixed membership stochastic blockmodels","authors":"Li","year":"2016","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Miller et al., 2009] Kurt Miller, Michael I Jordan, and Thomas L Griffiths. Non-parametric latent feature mod- els for link prediction. In Advances in neural information processing systems, pages 1276-1284, 2009.","title":"Non-parametric latent feature models for link prediction","authors":"Miller","year":"2009","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mørup et al., 2011] Morten Mørup, Mikkel N Schmidt, and Lars Kai Hansen. Infinite multiple membership relational modeling for complex networks. In Machine Learning for Signal Processing (MLSP), 2011 IEEE International Workshop on, pages 1-6. IEEE, 2011.","title":"Infinite multiple membership relational modeling for complex networks","authors":"Morten Mørup, N Mikkel, Lars Kai Schmidt, Hansen","year":"2011","venue":"Machine Learning for Signal Processing (MLSP)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nowicki and Snijders, 2001] Krzysztof Nowicki and Tom A B Snijders. Estimation and prediction for stochastic blockstructures. Journal of the American Statistical As- sociation, 96(455):1077-1087, 2001.","title":"Estimation and prediction for stochastic blockstructures","authors":"Krzysztof Nowicki, A B Tom, Snijders","year":"2001","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Roychowdhury et al., 2013] Anirban Roychowdhury, Ke Jiang, and Brian Kulis. Small-variance asymptotics for hidden markov models. In Advances in Neural Information Processing Systems, pages 2103-2111, 2013.","title":"Small-variance asymptotics for hidden markov models","authors":"Anirban Roychowdhury, Ke Jiang, Brian Kulis","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang and Zhu, 2015] Yining Wang and Jun Zhu. Dp-space: Bayesian nonparametric subspace clustering with small- variance asymptotics. In International Conference on Ma- chine Learning, pages 862-870, 2015.","title":"Dp-space: Bayesian nonparametric subspace clustering with smallvariance asymptotics","authors":"Wang, Zhu ; ] Yining Wang, Jun Zhu","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Xie et al., 2013] Jierui Xie, Stephen Kelley, and Boleslaw K Szymanski. Overlapping community detection in net- works: The state-of-the-art and comparative study. Acm computing surveys (csur), 45(4):43, 2013.","title":"Overlapping community detection in networks: The state-of-the-art and comparative study","authors":"Xie","year":"2013","venue":"Acm computing surveys (csur)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xu et al., 2015] Yanxun Xu, Peter Müller, Yuan Yuan, Ka- malakar Gulukota, and Yuan Ji. Mad bayes for tumor heterogeneity -feature allocation with exponential family sampling. Journal of the American Statistical Association, 110(510):503-514, 2015.","title":"Mad bayes for tumor heterogeneity -feature allocation with exponential family sampling","authors":"Xu","year":"2015","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhou, 2015] Mingyuan Zhou. Infinite edge partition models for overlapping community detection and link prediction. In AISTATS, 2015.","title":"Infinite edge partition models for overlapping community detection and link prediction","authors":"Mingyuan Zhou","year":"2015","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhu et al., 2016] Jun Zhu, Jiaming Song, and Bei Chen. Max-margin nonparametric latent feature models for link prediction. arXiv preprint arXiv:1602.07428, 2016.","title":"Max-margin nonparametric latent feature models for link prediction","authors":"Jun Zhu, Jiaming Song, Bei Chen","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}