{"type":"ScholarlyArticle","title":"Negative-Binomial Randomized Gamma Dynamical Systems for Heterogeneous Overdispersed Count Time Sequences","authors":[{"name":"Rui Huang","slug":"rui-huang","given":"Rui","family":"Huang","affiliation":"School of Computing and Information Technology , Great Bay University , 523000 Dongguan , China; Great Bay Institute for Advanced Study , Great Bay University; Guangdong Provincial Key Laboratory of Mathematical and Neural Dynamical Systems , Great Bay University; Dongguan Key Laboratory for Data Science and Intelligent Medicine , Great Bay University"},{"name":"Sikun Yang","slug":"sikun-yang","given":"Sikun","family":"Yang","affiliation":"School of Computing and Information Technology , Great Bay University , 523000 Dongguan , China; Great Bay Institute for Advanced Study , Great Bay University; Guangdong Provincial Key Laboratory of Mathematical and Neural Dynamical Systems , Great Bay University; Dongguan Key Laboratory for Data Science and Intelligent Medicine , Great Bay University"},{"name":"Heinz Koeppl","slug":"heinz-koeppl","given":"Heinz","family":"Koeppl","affiliation":"Department of Electrical Engineering and Information Technology , Technische Universität Darmstadt"}],"abstract":"Modeling count-valued time sequences has been receiving growing interests because count time sequences naturally arise in physical and social domains. Poisson gamma dynamical systems (PGDSs) are newly-developed methods, which can well capture the expressive latent transition structure and bursty dynamics behind count sequences. In particular, PGDSs demonstrate superior performance in terms of data imputation and prediction, compared with canonical linear dynamical system (LDS) based methods. Despite these advantages, PGDS cannot capture the heterogeneous overdispersed behaviours of the underlying dynamic processes. To mitigate this defect, we propose a negative-binomial-randomized gamma Markov process, which not only significantly improves the predictive performance of the proposed dynamical system, but also facilitates the fast convergence of the inference algorithm. Moreover, we develop methods to estimate both factor-structured and graph-structured transition dynamics, which enable us to infer more explainable latent structure, compared with PGDSs. Finally, we demonstrate the explainable latent structure learned by the proposed method, and show its superior performance in imputing missing data and forecasting future observations, compared with the related models.","doi":"10.24963/ijcai.2024/461","url":"https://ijcai.org/proceedings/2024/461","pdf":"https://ijcai.org/proceedings/2024/0461.pdf","markdown":"https://ijcai.org/proceedings/2024/461.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/461","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":461,"pages":{"first":4174,"last":4182},"keywords":["Uncertainty in AI: UAI: Tractable probabilistic models","Machine Learning: ML: Bayesian learning","Machine Learning: ML: Time series and data streams","Machine Learning: ML: Probabilistic machine learning"],"videos":[],"references":[{"position":1,"raw":"[Acharya et al., 2015] Ayan Acharya, Joydeep Ghosh, and Mingyuan Zhou. Nonparametric bayesian factor analysis for dynamic count matrices. In Artificial Intelligence and Statistics, pages 1-9. PMLR, 2015.","title":"Nonparametric bayesian factor analysis for dynamic count matrices","authors":"Ayan Acharya, Joydeep Ghosh, Mingyuan Zhou","year":"2015","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Acharya et al., 2018] Ayan Acharya, Joydeep Ghosh, and Mingyuan Zhou. A dual markov chain topic model for dynamic environments. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discov- ery & Data Mining, page 1099-1108, 2018.","title":"A dual markov chain topic model for dynamic environments","authors":"Ayan Acharya, Joydeep Ghosh, Mingyuan Zhou","year":"2018","venue":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Blei and Lafferty, 2006] David M Blei and John D Lafferty. Dynamic topic models. In Proceedings of the 23rd inter- national conference on Machine learning, pages 113-120, 2006.","title":"Dynamic topic models","authors":"M David, John D Blei, Lafferty","year":"2006","venue":"Proceedings of the 23rd international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chandra et al., 2023] Noirrit Kiran Chandra, Antonio Canale, and David B. Dunson. Escaping the curse of di- mensionality in bayesian model-based clustering. Journal of Machine Learning Research, 24(144):1-42, 2023.","title":"Escaping the curse of dimensionality in bayesian model-based clustering","authors":"Chandra","year":"2023","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2020] Wenchao Chen, Bo Chen, Yicheng Liu, Qianru Zhao, and Mingyuan Zhou. Switching poisson gamma dynamical systems. In Christian Bessiere, editor, Proceedings of the Twenty-Ninth International Joint Con- ference on Artificial Intelligence, IJCAI-20, pages 2029- 2036. International Joint Conferences on Artificial Intelli- gence Organization, 7 2020. Main track.","title":"Switching poisson gamma dynamical systems","authors":"Wenchao Chen, Bo Chen, Yicheng Liu, Qianru Zhao, Mingyuan Zhou","year":"2020","venue":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20","doi":null,"url":null,"internal_url":"/proceedings/2020/281"},{"position":6,"raw":"[Dieng et al., 2019] Adji B. Dieng, Francisco J. R. Ruiz, and David M. Blei. The dynamic embedded topic model. ArXiv, 2019.","title":"The dynamic embedded topic model","authors":"B Adji, Dieng, J R Francisco, David M Ruiz, Blei","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ferguson, 1973] Thomas S. Ferguson. A Bayesian Analysis of Some Nonparametric Problems. The Annals of Statis- tics, 1(2):209 -230, 1973.","title":"A Bayesian Analysis of Some Nonparametric Problems","authors":"Thomas S Ferguson","year":"1973","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Filstroff et al., 2021] Louis Filstroff, Olivier Gouvert, Cedric Fevotte, and Olivier Cappe. A comparative study of gamma markov chains for temporal non-negative matrix factorization. IEEE Transactions on Signal Processing, 69:1614 -1626, 2021.","title":"A comparative study of gamma markov chains for temporal non-negative matrix factorization","authors":"Filstroff","year":"2021","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gan et al., 2015] Zhe Gan, Chunyuan Li, Ricardo Henao, David Carlson, and Lawrence Carin. Deep temporal sig- moid belief networks for sequence modeling. In Proceed- ings of the 28th International Conference on Neural In- formation Processing Systems -Volume 2, NIPS'15, page 2467-2475, Cambridge, MA, USA, 2015. MIT Press.","title":"Deep temporal sigmoid belief networks for sequence modeling","authors":"Zhe Gan, Chunyuan Li, Ricardo Henao, David Carlson, Lawrence Carin","year":"2015","venue":"Proceedings of the 28th International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ghahramani and Roweis, 1998] Zoubin Ghahramani and Sam Roweis. Learning nonlinear dynamical systems using an em algorithm. Advances in neural information process- ing systems, 11:431-437, 1998.","title":"Learning nonlinear dynamical systems using an em algorithm","authors":"Roweis Ghahramani, Zoubin Ghahramani, Sam Roweis","year":"1998","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Han et al., 2014] Shaobo Han, Lin Du, Esther Salazar, and Lawrence Carin. Dynamic rank factor model for text streams. Advances in Neural Information Processing Sys- tems, 27:2663-2671, 2014.","title":"Dynamic rank factor model for text streams","authors":"Shaobo Han, Lin Du, Esther Salazar, Lawrence Carin","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jones et al., 2023] Andrew Jones, F. William Townes, Di- dong Li, and Barbara E. Engelhardt. Alignment of spa- tial genomics data using deep Gaussian processes. Nature Methods, 20(9):1379-1387, 2023.","title":"Alignment of spatial genomics data using deep Gaussian processes","authors":"Andrew Jones, F William Townes, Didong Li, Barbara E Engelhardt","year":"2023","venue":"Nature Methods","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[King, 2001] Gary King. Proper nouns and methodological propriety: Pooling dyads in international relations data. In- ternational Organization, 55(2):497-507, 2001.","title":"Proper nouns and methodological propriety: Pooling dyads in international relations data","authors":"Gary King","year":"2001","venue":"International Organization","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Levitin et al., 2019] Hanna Mendes Levitin, Jinzhou Yuan, Yim Ling Cheng, Francisco JR Ruiz, Erin C Bush, Jef- frey N Bruce, Peter Canoll, Antonio Iavarone, Anna La- sorella, David M Blei, et al. De novo gene signature identi- fication from single-cell rna-seq with hierarchical poisson factorization. Molecular systems biology, 15(2):e8557, 2019.","title":"De novo gene signature identification from single-cell rna-seq with hierarchical poisson factorization","authors":"Hanna Mendes Levitin, Jinzhou Yuan, Ling Yim, Francisco Jr Cheng, Erin C Ruiz, Jeffrey N Bush, Peter Bruce, Antonio Canoll, Anna Iavarone, David M Lasorella, Blei","year":"2019","venue":"Molecular systems biology","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Linderman et al., 2017] Scott Linderman, Matthew John- son, Andrew Miller, Ryan Adams, David Blei, and Liam Paninski. Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems. In Proceedings of the 20th International Conference on Artificial Intelli- gence and Statistics, pages 914-922, 2017.","title":"Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems","authors":"Scott Linderman, Matthew Johnson, Andrew Miller, Ryan Adams, David Blei, Liam Paninski","year":"2017","venue":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nassar et al., 2018] Josue Nassar, Scott Linderman, Monica Bugallo, and Il Memming Park. Tree-structured recurrent switching linear dynamical systems for multi-scale mod- eling. 2018.","title":"Tree-structured recurrent switching linear dynamical systems for multi-scale modeling","authors":"Josue Nassar, Scott Linderman, Monica Bugallo, Il Memming Park","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Roy and Dunson, 2020] Arkaprava Roy and David B Dun- son. Nonparametric graphical model for counts. The Journal of Machine Learning Research, 21(1):9353-9373, 2020.","title":"Arkaprava Roy and David B Dunson. Nonparametric graphical model for counts","authors":"Dunson Roy","year":"2020","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Rudolph and Blei, 2018] Maja Rudolph and David Blei. Dynamic embeddings for language evolution. In Pro- ceedings of the 2018 World Wide Web Conference, pages 1003--1011, 2018.","title":"Dynamic embeddings for language evolution","authors":"Rudolph, Blei ;, Maja Rudolph, David Blei","year":"2018","venue":"Proceedings of the 2018 World Wide Web Conference","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Schein et al., 2016a] Aaron Schein, Hanna Wallach, and Mingyuan Zhou. Poisson-gamma dynamical systems. Advances in Neural Information Processing Systems, 29:5012-5020, 2016.","title":"Poisson-gamma dynamical systems","authors":"Schein","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Schein et al., 2016b] Aaron Schein, Mingyuan Zhou, David Blei, and Hanna Wallach. Bayesian poisson tucker de- composition for learning the structure of international re- lations. In Proceedings of The 33rd International Confer- ence on Machine Learning, pages 2810-2819, 2016.","title":"Bayesian poisson tucker decomposition for learning the structure of international relations","authors":"Schein","year":"2016","venue":"Proceedings of The 33rd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Schein et al., 2019] Aaron Schein, Scott Linderman, Mingyuan Zhou, David Blei, and Hanna Wallach. Poisson-randomized gamma dynamical systems. Ad- vances in Neural Information Processing Systems, 32:782-793, 2019.","title":"Poisson-randomized gamma dynamical systems","authors":"Aaron Schein, Scott Linderman, Mingyuan Zhou, David Blei, Hanna Wallach","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sheldon et al., 2013] Daniel Sheldon, Tao Sun, Akshat Ku- mar, and Tom Dietterich. Approximate inference in col- lective graphical models. In Proceedings of the 30th Inter- national Conference on Machine Learning, pages 1004- 1012, 2013.","title":"Approximate inference in collective graphical models","authors":"Daniel Sheldon, Tao Sun, Akshat Kumar, Tom Dietterich","year":"2013","venue":"Proceedings of the 30th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Stewart, 2014] Brandon M Stewart. Latent factor regres- sions for the social sciences. Technical report, Harvard University, 2014.","title":"Latent factor regressions for the social sciences","authors":"M Brandon, Stewart","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Stuart and Wolfram, 2020] Andrew M. Stuart and Marie- Therese Wolfram. Inverse optimal transport. SIAM Jour- nal on Applied Mathematics, 80(1):599-619, 2020.","title":"Inverse optimal transport","authors":"Andrew M Stuart, Marie-Therese Wolfram","year":"2020","venue":"SIAM Journal on Applied Mathematics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tong et al., 2020] Alexander Tong, Jessie Huang, Guy Wolf, David Van Dijk, and Smita Krishnaswamy. Trajec- torynet: A dynamic optimal transport network for model- ing cellular dynamics. In Proceedings of the 37th Interna- tional Conference on Machine Learning, 2020.","title":"Trajectorynet: A dynamic optimal transport network for modeling cellular dynamics","authors":"Alexander Tong, Jessie Huang, Guy Wolf, David Van Dijk, Smita Krishnaswamy","year":"2020","venue":"Proceedings of the 37th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Virtanen and Girolami, 2020] Seppo Virtanen and Mark Girolami. Dynamic content based ranking. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, pages 2315-2324, 2020.","title":"Dynamic content based ranking","authors":"Girolami Virtanen, Seppo Virtanen, Mark Girolami","year":"2020","venue":"Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2008] Chong Wang, David M. Blei, and David E. Heckerman. Continuous time dynamic topic models. In Conference on Uncertainty in Artificial Intelli- gence, 2008.","title":"Continuous time dynamic topic models","authors":"Chong Wang, David M Blei, David E Heckerman","year":"2008","venue":"Conference on Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou and Carin, 2012] Mingyuan Zhou and Lawrence Carin. Augment-and-conquer negative binomial pro- cesses. In Advances in Neural Information Processing Systems, volume 25, pages 2546-2554.","title":"Augment-and-conquer negative binomial processes","authors":"Carin ; Zhou, Mingyuan Zhou, Lawrence Carin","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"Curran Associates, Inc., 2012.","title":null,"authors":"Curran Associates, Inc","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhou and Carin, 2015] Mingyuan Zhou and Lawrence Carin. Negative binomial process count and mixture modeling. IEEE Trans. Pattern Anal. Mach. Intell., 37(2):307-320, 2015.","title":"Negative binomial process count and mixture modeling","authors":"Carin ; Zhou, Mingyuan Zhou, Lawrence Carin","year":"2015","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhou et al., 2016] Mingyuan Zhou, Yulai Cong, and Bo Chen. Augmentable gamma belief networks. Journal of Machine Learning Research, 17(163):1-44, 2016.","title":"Augmentable gamma belief networks","authors":"Mingyuan Zhou, Yulai Cong, Bo Chen","year":"2016","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhou, 2015] Mingyuan Zhou. Infinite Edge Partition Mod- els for Overlapping Community Detection and Link Pre- diction. In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, pages 1135-1143, 2015.","title":"Infinite Edge Partition Models for Overlapping Community Detection and Link Prediction","authors":"Mingyuan Zhou","year":"2015","venue":"Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhou, 2016a] Mingyuan Zhou. Nonparametric bayesian negative binomial factor analysis. Bayesian Analysis, 13(4):1065-1093, 2016.","title":"Nonparametric bayesian negative binomial factor analysis","authors":"Zhou ; A] Mingyuan, Zhou","year":"2016","venue":"Bayesian Analysis","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhou, 2016b] Mingyuan Zhou. Softplus regressions and convex polytopes, 2016.","title":"Softplus regressions and convex polytopes","authors":"Mingyuan Zhou, Zhou","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhou, 2018] Mingyuan Zhou. Parsimonious bayesian deep networks. In Advances in Neural Information Processing Systems, 2018.","title":"Parsimonious bayesian deep networks","authors":"Mingyuan Zhou","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}