{"type":"ScholarlyArticle","title":"Switching Poisson Gamma Dynamical Systems","authors":[{"name":"Wenchao Chen","slug":"wenchao-chen","given":"Wenchao","family":"Chen","affiliation":"National Laboratory of Radar Signal Processing , Xidian University , Xian , China"},{"name":"Bo Chen","slug":"bo-chen","given":"Bo","family":"Chen","affiliation":"National Laboratory of Radar Signal Processing , Xidian University , Xian , China"},{"name":"Yicheng Liu","slug":"yicheng-liu","given":"Yicheng","family":"Liu","affiliation":"National Laboratory of Radar Signal Processing , Xidian University , Xian , China"},{"name":"Qianru Zhao","slug":"qianru-zhao","given":"Qianru","family":"Zhao","affiliation":"National Laboratory of Radar Signal Processing , Xidian University , Xian , China"},{"name":"Mingyuan Zhou","slug":"mingyuan-zhou","given":"Mingyuan","family":"Zhou","affiliation":"McCombs School of Business , The University of Texas at Austin , Austin , TX 78712 , USA"}],"abstract":"We propose Switching Poisson gamma dynamical systems (SPGDS) to model sequentially observed multivariate count data. Different from previous models, SPGDS assigns its latent variables into mixture of gamma distributed parameters to model complex sequences and describe the nonlinear dynamics, meanwhile,  capture various temporal dependencies. For efficient inference, we develop a scalable hybrid stochastic gradient-MCMC and switching recurrent autoencoding variational inference, which is scalable to large scale sequences and fast in out-of-sample prediction. Experiments on both unsupervised and supervised tasks demonstrate that the proposed model not only has excellent fitting and prediction performance on complex dynamic sequences, but also separates different dynamical patterns within them.","doi":"10.24963/ijcai.2020/281","url":"https://ijcai.org/proceedings/2020/281","pdf":"https://ijcai.org/proceedings/2020/0281.pdf","markdown":"https://ijcai.org/proceedings/2020/281.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/281","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":281,"pages":{"first":2029,"last":2036},"keywords":["Machine Learning: Deep Learning: Sequence Modeling","Machine Learning: Probabilistic Machine Learning","Machine Learning: Learning Generative Models"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24209/file_24209.m3u8","length_sec":275,"provider":"hls"}],"references":[{"position":1,"raw":"[Aaron et al., 2019] Schein Aaron, W. Linderman Scott, Zhou Mingyuan, M. Blei David, and M. Wallach Han- na. Poisson-randomized gamma dynamical systems. In NeurIPS, pages 781-792, 2019.","title":"Poisson-randomized gamma dynamical systems","authors":"W Linderman Schein Aaron, Scott, M Blei Zhou Mingyuan, M Wallach David, Hanna","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Acharya et al., ] Ayan Acharya, Joydeep Ghosh, and Mingyuan Zhou. Nonparametric Bayesian factor analy- sis for dynamic count matrices. In AISTATS, pages 1462- 1471.","title":"Nonparametric Bayesian factor analysis for dynamic count matrices","authors":"Acharya","year":null,"venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Arjovsky et al., 2016] Martin Arjovsky, Amar Shah, and Yoshua Bengio. Unitary evolution recurrent neural net- works. In ICML, pages 1120-1128, 2016.","title":"Unitary evolution recurrent neural networks","authors":"Martin Arjovsky, Amar Shah, Yoshua Bengio","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Barone, 2016] Antonio Valerio Miceli Barone. Low-rank passthrough neural networks. pages 77-86, 2016.","title":"Low-rank passthrough neural networks","authors":"Antonio Valerio, Miceli Barone","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Becker Ehmck et al., 2019] Philip Becker Ehmck, Jan Pe- ters, and van der Smagt Patrick. Switching linear dynamic- s for variational bayes filtering. In ICML, pages 553-562, 2019.","title":"Philip Becker Ehmck, Jan Peters, and van der Smagt Patrick. Switching linear dynamics for variational bayes filtering","authors":"Becker Ehmck","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Campos et al., 2018] Victor Campos, Brendan Jou, Xavier Giro-I-Nieto, Jordi Torres, and Shih Fu Chang. Skip rnn: Learning to skip state updates in recurrent neural network- s. In ICLR, 2018.","title":"Skip rnn: Learning to skip state updates in recurrent neural networks","authors":"Victor Campos, Brendan Jou, Xavier Giro-I-Nieto, Jordi Torres, Shih Fu, Chang","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart Van Merrienboer, Caglar Gulcehre, Fethi Bougares, and Yoshua Ben- gio. Learning phrase representations using rnn encoder- decoder for statistical machine translation. Computer Sci- ence, pages 1724-1734, 2014.","title":"Learning phrase representations using rnn encoderdecoder for statistical machine translation","authors":"Cho","year":"2014","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chung et al., 2015] Junyoung Chung, Kyle Kastner, Lau- rent Dinh, Kratarth Goel, Aaron Courville, and Yoshua Bengio. A recurrent latent variable model for sequential data. In NeurIPS, pages 2980-2988, 2015.","title":"A recurrent latent variable model for sequential data","authors":"Chung","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cong et al., 2017] Yulai Cong, Chen Bo, Hongwei Liu, and Mingyuan Zhou. Deep latent dirichlet allocation with topic-layer-adaptive stochastic gradient riemannian mcmc. In ICML, pages 864-873, 2017.","title":"Deep latent dirichlet allocation with topic-layer-adaptive stochastic gradient riemannian mcmc","authors":"Cong","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fraccaro et al., 2017] Marco Fraccaro, Simon Kamronn, Ulrich Paquet, and Ole Winther. A disentangled recogni- tion and nonlinear dynamics model for unsupervised learn- ing. In NeurIPS, pages 3601-3610, 2017.","title":"A disentangled recognition and nonlinear dynamics model for unsupervised learning","authors":"Marco Fraccaro, Simon Kamronn, Ulrich Paquet, Ole Winther","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gan et al., 2015] Z Gan, R Henao, D Carlson, and L Carin. Learning deep sigmoid belief networks with data augmen- tation. In AISTATS, 2015.","title":"Learning deep sigmoid belief networks with data augmentation","authors":"R Gan, D Henao, L Carlson, Carin","year":"2015","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ghahramani and Roweis, 1999] Zoubin Ghahramani and Sam T. Roweis. Learning nonlinear dynamical systems using an em algorithm. In NeurIPS, 11:431-437, 1999.","title":"Learning nonlinear dynamical systems using an em algorithm","authors":"Roweis ; ] Zoubin Ghahramani, Sam T Ghahramani, Roweis","year":"1999","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gong and Huang, 2017] C. Y Gong and W Huang. Deep dynamic poisson factorization model. In neurIPS, pages 1666-1674, 2017.","title":"Deep dynamic poisson factorization model","authors":"C Y Gong, W Huang","year":"2017","venue":"neurIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Greff et al., 2016] K Greff, R. K. Srivastava, J Koutnik, B. R. Steunebrink, and J Schmidhuber. Lstm: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems, 28(10):2222-2232, 2016.","title":"Lstm: A search space odyssey","authors":"R K Greff, Srivastava, B R Koutnik, Steunebrink, Schmidhuber","year":"2016","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Guo et al., 2018] Dandan Guo, Bo Chen, Hao Zhang, and Mingyuan Zhou. Deep poisson gamma dynamical system- s. In NeurIPS, pages 8451-8461, 2018.","title":"Deep poisson gamma dynamical systems","authors":"Dandan Guo, Bo Chen, Hao Zhang, Mingyuan Zhou","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Han et al., 2014] S Han, L Du, E Salazar, and L Carin. Dy- namic rank factor model for text streams. In NeurIPS, pages 2663-2671, 2014.","title":"Dynamic rank factor model for text streams","authors":"L Han, E Du, L Salazar, Carin","year":"2014","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jang et al., 2016] Eric Jang, Gu Shixiang, and Poole Ben. Categorical reparameterization with gumbel- softmax. arXiv preprint: 1611.01144, 2016.","title":"Categorical reparameterization with gumbelsoftmax","authors":"Eric Jang, Gu Shixiang, Poole Ben","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Krishnan et al., 2017] Rahul G. Krishnan, Uri Shalit, and David A. Sontag. Structured inference networks for non- linear state space models. In AAAI, pages 2101-2109, 2017.","title":"Structured inference networks for nonlinear state space models","authors":"G Rahul, Uri Krishnan, David A Shalit, Sontag","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Le et al., 2015] Quoc V. Le, Navdeep Jaitly, and Geof- frey E. Hinton. A simple way to initialize recurrent net- works of rectified linear units. Computer Science, 2015.","title":"A simple way to initialize recurrent networks of rectified linear units","authors":"V Quoc, Navdeep Le, Geoffrey E Jaitly, Hinton","year":"2015","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[LeCun et al., 1998] Y LeCun, L Bottou, Y Bengio, and P Haffner. Gradient based learning applied to documen- t recognition. Proceedings of the IEEE, 86:2278-2324, 1998.","title":"Gradient based learning applied to document recognition","authors":"Lecun, Y Bottou, Bengio, Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Linderman et al., 2016] Scott W. Linderman, Andrew C. Miller, and Ryan P. Adams. Recurrent switching linear dynamical systems. In AISTATS, 2016.","title":"Recurrent switching linear dynamical systems","authors":"Scott W Linderman, Andrew C Miller, Ryan P Adams","year":"2016","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ma et al., 2015] Yi An Ma, Tianqi Chen, and Emily B. Fox. A complete recipe for stochastic gradient mcmc. In NeurIPS, pages 2917-2925, 2015.","title":"A complete recipe for stochastic gradient mcmc","authors":"An Yi, Tianqi Ma, Emily B Chen, Fox","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Maddison et al., 2016] Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. CoRR, 2016. [Martens and Sutskever, 2011] James Martens and Ilya Sutskever. Learning recurrent neural networks with Hessian-Free optimization. In ICML, pages 1033-1040, 2011.","title":"The concrete distribution: A continuous relaxation of discrete random variables","authors":"Andriy Chris J Maddison, Yee Whye Mnih, Teh","year":"2011","venue":"Learning recurrent neural networks with Hessian-Free optimization","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Patterson and W, 2013] S Patterson and Teh Y. W. Stochas- tic gradient riemannian langevin dynamics on the proba- bility simplex. In NeurIPS, pages 3102-3110, 2013.","title":"Stochastic gradient riemannian langevin dynamics on the probability simplex","authors":"W ; ] S Patterson, Patterson, Y W Teh","year":"2013","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Rabiner and Juang, 1986] L Rabiner and B Juang. An intro- duction to Hidden Markov Models. IEEE ASSP magazine, 1986.","title":"An introduction to Hidden Markov Models","authors":"L Rabiner, Juang","year":"1986","venue":"IEEE ASSP magazine","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schein et al., 2016] Aaron Schein, Mingyuan Zhou, and Hanna Wallach. Poisson-gamma dynamical systems. In NeurIPS, pages 5006-5014, 2016.","title":"Poisson-gamma dynamical systems","authors":"Aaron Schein, Mingyuan Zhou, Hanna Wallach","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2019] Chaojie Wang, Bo Chen, Shucheng Xi- ao, and Mingyuan Zhou. Convolutional poisson gamma belief network. In ICML, pages 6515-6525, 2019.","title":"Convolutional poisson gamma belief network","authors":"Wang","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Welling and Teh, 2011] Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient langevin dynam- ics. In ICML, pages 681-688, 2011.","title":"Bayesian learning via stochastic gradient langevin dynamics","authors":"Teh ; ] Max Welling, Yee Whye Welling, Teh","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wisdom et al., 2016] Scott Wisdom, Thomas Powers, John R. Hershey, Jonathan Le Roux, and Les Atlas. Full- capacity unitary recurrent neural networks. In NeurIPS, pages 4880-4888, 2016.","title":"Fullcapacity unitary recurrent neural networks","authors":"Scott Wisdom, Thomas Powers, John R Hershey, Jonathan Le Roux, Les Atlas","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2016] Saizheng Zhang, Yuhuai Wu, Che Tong, Zhouhan Lin, and Yoshua Bengio. Architectural complex- ity measures of recurrent neural networks. In NeurIPS, pages 1822-1830, 2016.","title":"Architectural complexity measures of recurrent neural networks","authors":"Zhang","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2018] Hao Zhang, Bo Chen, Dandan Guo, and Mingyuan Zhou. Whai: Weibull hybrid autoencoding in- ference for deep topic modeling. In ICLR, 2018.","title":"Whai: Weibull hybrid autoencoding inference for deep topic modeling","authors":"Hao Zhang, Bo Chen, Dandan Guo, Mingyuan Zhou","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhe et al., 2015] Gan Zhe, Chunyuan Li, Ricardo Henao, David Carlson, and Lawrence Carin. Deep temporal sig- moid belief networks for sequence modeling. In NeurIPS, pages 2467-2475, 2015.","title":"Deep temporal sigmoid belief networks for sequence modeling","authors":"Chunyuan Gan Zhe, Ricardo Li, David Henao, Lawrence Carlson, Carin","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhou et al., 2012] Mingyuan Zhou, Lauren Hannah, David Dunson, and Lawrence Carin. Beta-negative binomial pro- cess and poisson factor analysis. In AISTATS, pages 1462- 1471, 2012.","title":"Beta-negative binomial process and poisson factor analysis","authors":"Mingyuan Zhou, Lauren Hannah, David Dunson, Lawrence Carin","year":"2012","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhou et al., 2016] M. Zhou, Y. Cong, and B Chen. Aug- mentable gamma belief networks. Journal of Machine Learning Research, 17(163):1-44, 2016.","title":"Augmentable gamma belief networks","authors":"M Zhou, Y Cong, Chen","year":"2016","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhou, 2015] Mingyuan Zhou. Infinite edge partition models for overlapping community detection and link prediction. In AISTATS, pages 1135-1143, 2015.","title":"Infinite edge partition models for overlapping community detection and link prediction","authors":"Mingyuan Zhou","year":"2015","venue":"AISTATS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}