{"type":"ScholarlyArticle","title":"A Scalable Community Detection Algorithm for Large Graphs Using Stochastic Block Models","authors":[{"name":"Chengbin Peng","slug":"chengbin-peng","given":"Chengbin","family":"Peng","affiliation":""},{"name":"Zhihua Zhang","slug":"zhihua-zhang","given":"Zhihua","family":"Zhang","affiliation":""},{"name":"Ka-Chun Wong","slug":"ka-chun-wong","given":"Ka-Chun","family":"Wong","affiliation":""},{"name":"Xiangliang Zhang","slug":"xiangliang-zhang","given":"Xiangliang","family":"Zhang","affiliation":""},{"name":"David E. Keyes","slug":"david-e-keyes","given":"David E.","family":"Keyes","affiliation":""}],"abstract":"Community detection in graphs is widely used in social and biological networks, and the stochastic block model is a powerful probabilistic tool for describing graphs with community structures. However, in the era of ''big data,'' traditional inference algorithms for such a model are increasingly limited due to their high time complexity and poor scalability. In this paper, we propose a multi-stage maximum likelihood approach to recover the latent parameters of the stochastic block model, in time linear with respect to the number of edges. We also propose a parallel algorithm based on message passing. Our algorithm can overlap communication and computation, providing speedup without compromising accuracy as the number of processors grows. For example, to process a real-world graph with about 1.3 million nodes and 10 million edges, our algorithm requires about 6 seconds on 64 cores of a contemporary commodity Linux cluster. Experiments demonstrate that the algorithm can produce high quality results on both benchmark and real-world graphs. An example of finding more meaningful communities is illustrated consequently in comparison with a popular modularity maximization algorithm.","doi":null,"url":"https://ijcai.org/proceedings/2015/296","pdf":"https://ijcai.org/proceedings/2015/0296.pdf","markdown":"https://ijcai.org/proceedings/2015/296.md","bibtex":"https://ijcai.org/proceedings/2015/bibtex/296","volume":{"slug":"2015","title":"Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2015","url":"https://ijcai.org/proceedings/2015/","year":2015,"location":"Buenos Aires, Argentina","dates":"25–31 July 2015","editors":"Qiang Yang and Michael Wooldridge","isbn":"978-1-57735-738-4","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Social Networks","track":"Main Track","number":296,"pages":{"first":2090,"last":2095},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}