{"type":"ScholarlyArticle","title":"Recurrent Dirichlet Belief Networks for interpretable Dynamic Relational Data Modelling","authors":[{"name":"Yaqiong Li","slug":"yaqiong-li","given":"Yaqiong","family":"Li","affiliation":"Centre for Artificial Intelligence , University of Technology Sydney"},{"name":"Xuhui Fan","slug":"xuhui-fan","given":"Xuhui","family":"Fan","affiliation":"School of Mathematics & Statistics , University of New South Wales , Sydney"},{"name":"Ling Chen","slug":"ling-chen","given":"Ling","family":"Chen","affiliation":"Centre for Artificial Intelligence , University of Technology Sydney"},{"name":"Bin Li","slug":"bin-li","given":"Bin","family":"Li","affiliation":"School of Computer Science , Fudan University"},{"name":"Zheng Yu","slug":"zheng-yu","given":"Zheng","family":"Yu","affiliation":"Department of Electrical and Computer Engineering , University of Alberta"},{"name":"Scott A. Sisson","slug":"scott-a-sisson","given":"Scott A.","family":"Sisson","affiliation":"School of Mathematics & Statistics , University of New South Wales , Sydney"}],"abstract":"The Dirichlet Belief Network~(DirBN) has been recently proposed as a promising approach in learning interpretable deep latent representations for objects. \r\nIn this work, we leverage its interpretable modelling architecture and propose a deep dynamic probabilistic framework -- the Recurrent Dirichlet Belief Network~(Recurrent-DBN) -- to study interpretable hidden structures from dynamic relational data. The proposed Recurrent-DBN has the following merits: (1) it infers interpretable and organised hierarchical latent structures for objects within and across time steps; (2) it enables recurrent long-term temporal dependence modelling, which outperforms the one-order Markov descriptions in most of the dynamic probabilistic frameworks; (3) the computational cost scales to the number of positive links only. In addition, we develop a new inference strategy, which first upward-and-backward propagates latent counts and then downward-and-forward samples variables, to enable efficient Gibbs sampling for the Recurrent-DBN. We apply the Recurrent-DBN to dynamic relational data problems. The extensive experiment results on real-world data validate the advantages of the Recurrent-DBN over the state-of-the-art models in interpretable latent structure discovery and improved link prediction performance.","doi":"10.24963/ijcai.2020/342","url":"https://ijcai.org/proceedings/2020/342","pdf":"https://ijcai.org/proceedings/2020/0342.pdf","markdown":"https://ijcai.org/proceedings/2020/342.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/342","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":342,"pages":{"first":2470,"last":2476},"keywords":["Machine Learning: Deep Generative Models","Machine Learning: Probabilistic Machine Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24961/file_24961.m3u8","length_sec":311,"provider":"hls"}],"references":[{"position":1,"raw":"[Airoldi et al., 2008] Edoardo M. Airoldi, David M. Blei, Stephen E. Fienberg, and Eric P. Xing. Mixed membership stochastic block models. Journal of Machine Learning Re- search, 9:1981-2014, 2008.","title":"Mixed membership stochastic block models","authors":"M Edoardo, David M Airoldi, Stephen E Blei, Eric P Fienberg, Xing","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Coleman, 1964] James S. Coleman. Introduction to math- ematical sociology. Introduction to mathematical sociol- ogy, 1964.","title":"Introduction to mathematical sociology","authors":"James S Coleman","year":"1964","venue":"Introduction to mathematical sociology","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dunson and Herring, 2005] David B. Dunson and Amy H. Herring. Bayesian latent variable models for mixed dis- crete outcomes. Biostatistics, 6(1):11-25, 2005.","title":"Bayesian latent variable models for mixed discrete outcomes","authors":"David B Dunson, Amy H Herring","year":"2005","venue":"Biostatistics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Eagle and Pentland, 2006] Nathan Eagle and Alex Sandy Pentland. Reality mining: sensing complex social sys- tems. Personal and Ubiquitous Computing, 10(4):255- 268, 2006.","title":"Reality mining: sensing complex social systems","authors":"Pentland Eagle, Nathan Eagle, Alex Sandy, Pentland","year":"2006","venue":"Personal and Ubiquitous Computing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fan et al., 2014] Xuhui Fan, Longbing Cao, and Richard Yi Da Xu. Dynamic infinite mixed-membership stochastic block model. IEEE Transactions on Neural Networks and Learning Systems, 26(9):2072-2085, 2014.","title":"Dynamic infinite mixed-membership stochastic block model","authors":"Xuhui Fan, Longbing Cao, Richard Yi, Da Xu","year":"2014","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fan et al., 2016] Xuhui Fan, Bin Li, Yi Wang, Yang Wang, and Fang Chen. The Ostomachion Process. In AAAI, pages 1547-1553, 2016.","title":"The Ostomachion Process","authors":"Xuhui Fan, Bin Li, Yi Wang, Yang Wang, Fang Chen","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fan et al., 2018a] Xuhui Fan, Bin Li, and Scott Sisson. Rectangular bounding process. In NeurIPS, pages 7631- 7641, 2018.","title":"Rectangular bounding process","authors":"Fan","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fan et al., 2018b] Xuhui Fan, Bin Li, and Scott A. Sisson. The binary space partitioning-tree process. In AISTATS, volume 84, pages 1859-1867, 2018.","title":"The binary space partitioning-tree process","authors":"Fan","year":"2018","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fan et al., 2019a] Xuhui Fan, Bin Li, Caoyuan Li, Scott SIs- son, and Ling Chen. Scalable deep generative relational model with high-order node dependence. In NeurIPS, pages 12637-12647, 2019.","title":"Scalable deep generative relational model with high-order node dependence","authors":"Fan","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fan et al., 2019b] Xuhui Fan, Bin Li, and Scott A. Sisson. The binary space partitioning forests. In AISTATS, vol- ume 89, pages 3022-3031, 2019.","title":"The binary space partitioning forests","authors":"Fan","year":"2019","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fan et al., 2020] Xuhui Fan, Bin Li, and Scott A. Sisson. Online binary space partitioning forests. In AISTATS, 2020.","title":"Online binary space partitioning forests","authors":"Xuhui Fan, Bin Li, Scott A Sisson","year":"2020","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gan et al., 2015] Zhe Gan, Chunyuan Li, Ricardo Henao, David E Carlson, and Lawrence Carin. Deep temporal sig- moid belief networks for sequence modeling. In NeurIPS, pages 2467-2475, 2015.","title":"Deep temporal sigmoid belief networks for sequence modeling","authors":"Zhe Gan, Chunyuan Li, Ricardo Henao, David E Carlson, Lawrence Carin","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gong, 2017] Chengyue Gong. Deep dynamic poisson fac- torization model. In NeurIPS, pages 1666-1674, 2017.","title":"Chengyue Gong. Deep dynamic poisson factorization model","authors":"Gong","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Guo et al., 2018] Dandan Guo, Bo Chen, Hao Zhang, and Mingyuan Zhou. Deep poisson gamma dynamical sys- tems. In NeurIPS, pages 8442-8452, 2018.","title":"Deep poisson gamma dynamical systems","authors":"Dandan Guo, Bo Chen, Hao Zhang, Mingyuan Zhou","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Henao et al., 2015] Ricardo Henao, Zhe Gan, James Lu, and Lawrence Carin. Deep poisson factor modeling. In NeurIPS, pages 2800-2808, 2015.","title":"Deep poisson factor modeling","authors":"Ricardo Henao, Zhe Gan, James Lu, Lawrence Carin","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Isella et al., 2011] Lorenzo Isella, Juliette Stehlé, Alain Barrat, Ciro Cattuto, Jean-Franc ¸ois Pinton, and Wouter Van den Broeck. What's in a crowd? analysis of face-to- face behavioral networks. Journal of Theoretical Biology, 271(1):166-180, 2011.","title":"What's in a crowd? analysis of face-toface behavioral networks","authors":"Lorenzo Isella, Juliette Stehlé, Alain Barrat, Ciro Cattuto, Jean-Franc ¸ois Pinton, Wouter Van Den Broeck","year":"2011","venue":"Journal of Theoretical Biology","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Koren, 2009] Yehuda Koren. Collaborative filtering with temporal dynamics. In KDD, pages 447-456, 2009.","title":"Collaborative filtering with temporal dynamics","authors":"Koren ;, Yehuda Koren","year":"2009","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lloyd et al., 2012] James Lloyd, Peter Orbanz, Zoubin Ghahramani, and Daniel M Roy. Random function priors for exchangeable arrays with applications to graphs and relational data. In NIPS, pages 1007-1015, 2012.","title":"Random function priors for exchangeable arrays with applications to graphs and relational data","authors":"Lloyd","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mucha et al., 2010] Peter J. Mucha, Thomas Richardson, Kevin Macon, Mason A Porter, and Jukka-Pekka Onnela. Community structure in time-dependent, multiscale, and multiplex networks. Science, 328(5980):876-878, 2010.","title":"Community structure in time-dependent, multiscale, and multiplex networks","authors":"J Peter, Thomas Mucha, Kevin Richardson, Mason A Macon, Jukka-Pekka Porter, Onnela","year":"2010","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Newman, 2001] Mark EJ. Newman. Clustering and prefer- ential attachment in growing networks. Physical Review E, 64(2):025102, 2001.","title":"Clustering and preferential attachment in growing networks","authors":"E J Mark, Newman","year":"2001","venue":"Physical Review E","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Orbanz and Roy, 2014] Peter Orbanz and Daniel M Roy. Bayesian models of graphs, arrays and other exchangeable random structures. IEEE transactions on pattern analysis and machine intelligence, 37(2):437-461, 2014.","title":"Peter Orbanz and Daniel M Roy. Bayesian models of graphs, arrays and other exchangeable random structures","authors":"Roy Orbanz","year":"2014","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social rep- resentations. In KDD, pages 701-710, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Phan and Airoldi, 2015] Tuan Q. Phan and Edoardo M. Airoldi. A natural experiment of social network forma- tion and dynamics. Proceedings of the National Academy of Sciences, 112(21):6595-6600, 2015.","title":"A natural experiment of social network formation and dynamics","authors":"Tuan Q Phan, Edoardo M Airoldi","year":"2015","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Schein et al., 2015] Aaron Schein, John Paisley, David M. Blei, and Hanna Wallach. Bayesian poisson tensor fac- torization for inferring multilateral relations from sparse dyadic event counts. In KDD, pages 1045-1054, 2015.","title":"Bayesian poisson tensor factorization for inferring multilateral relations from sparse dyadic event counts","authors":"Aaron Schein, John Paisley, David M Blei, Hanna Wallach","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Schein et al., 2016] Aaron Schein, Hanna Wallach, and Mingyuan Zhou. Poisson-gamma dynamical systems. In NeurIPS, pages 5005-5013, 2016.","title":"Poisson-gamma dynamical systems","authors":"Aaron Schein, Hanna Wallach, Mingyuan Zhou","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schein et al., 2019] Aaron Schein, Scott Linderman, Mingyuan Zhou, David Blei, and Hanna Wallach. Poisson-randomized gamma dynamical systems. In NeurIPS, pages 781-792, 2019.","title":"Poisson-randomized gamma dynamical systems","authors":"Aaron Schein, Scott Linderman, Mingyuan Zhou, David Blei, Hanna Wallach","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tarrés-Deulofeu et al., 2019] Marc Tarrés-Deulofeu, Anto- nia Godoy-Lorite, Roger Guimerà, and Marta Sales-Pardo. Tensorial and bipartite block models for link prediction in layered networks and temporal networks. Physical Review E, 99(3):032307, 2019.","title":"Tensorial and bipartite block models for link prediction in layered networks and temporal networks","authors":"Tarrés-Deulofeu","year":"2019","venue":"Physical Review E","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang and Koeppl, 2018] Sikun Yang and Heinz Koeppl. Dependent relational gamma process models for longitu- dinal networks. In ICML, pages 5547-5556, 2018.","title":"Dependent relational gamma process models for longitudinal networks","authors":"Yang, Koeppl ; ] Sikun Yang, Heinz Koeppl","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yu and Fan, 2020] Zheng Yu and Xuhui Fan. Fragmentation coagulation based mixed-membership stochastic block- model. Arxiv, 2020.","title":"Fragmentation coagulation based mixed-membership stochastic blockmodel","authors":"Zheng Yu, Xuhui Fan","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2018] Hongming Zhang, Liwei Qiu, Lingling Yi, and Yangqiu Song. Scalable multiplex network em- bedding. In IJCAI, volume 18, pages 3082-3088, 2018.","title":"Scalable multiplex network embedding","authors":"Hongming Zhang, Liwei Qiu, Lingling Yi, Yangqiu Song","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/428"},{"position":31,"raw":"[Zhao et al., 2018] He Zhao, Lan Du, Wray Buntine, and Mingyuan Zhou. Dirichlet belief networks for topic struc- ture learning. In NeurIPS, pages 7955-7966, 2018.","title":"Dirichlet belief networks for topic structure learning","authors":"Zhao","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhou, 2015] Mingyuan Zhou. Infinite edge partition models for overlapping community detection and link prediction. In AISTATS, pages 1135-1143, 2015.","title":"Infinite edge partition models for overlapping community detection and link prediction","authors":"Mingyuan Zhou","year":"2015","venue":"AISTATS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}