{"type":"ScholarlyArticle","title":"Learning Mixture of Neural Temporal Point Processes for Multi-dimensional Event Sequence Clustering","authors":[{"name":"Yunhao Zhang","slug":"yunhao-zhang","given":"Yunhao","family":"Zhang","affiliation":"Department of CSE , MoE Key Lab of Artificial Intelligence , Shanghai Jiao Tong University"},{"name":"Junchi Yan","slug":"junchi-yan","given":"Junchi","family":"Yan","affiliation":"Department of CSE , MoE Key Lab of Artificial Intelligence , Shanghai Jiao Tong University"},{"name":"Xiaolu Zhang","slug":"xiaolu-zhang","given":"Xiaolu","family":"Zhang","affiliation":"Ant Financial Services Group"},{"name":"Jun Zhou","slug":"jun-zhou","given":"Jun","family":"Zhou","affiliation":"Ant Financial Services Group"},{"name":"Xiaokang Yang","slug":"xiaokang-yang","given":"Xiaokang","family":"Yang","affiliation":"Department of CSE , MoE Key Lab of Artificial Intelligence , Shanghai Jiao Tong University"}],"abstract":"Multi-dimensional event sequence clustering applies to many scenarios e.g. e-Commerce and electronic health. Traditional clustering models fail to characterize complex real-world processes due to the strong parametric assumption. While Neural Temporal Point Processes (NTPPs) mainly focus on modeling similar sequences instead of clustering. To fill the gap, we propose Mixture of Neural Temporal Point Processes (NTPP-MIX), a  general framework that can utilize many existing NTPPs for multi-dimensional event sequence clustering. In NTPP-MIX, the prior distribution of coefficients for cluster assignment is modeled by a Dirichlet distribution. When the assignment is given, the conditional probability of a sequence is modeled by the mixture of a series of NTPPs. We combine variational EM algorithm and Stochastic Gradient Descent (SGD) to efficiently train the framework. In E-step, we fix parameters for NTPPs and approximate the true posterior with variational distributions. In M-step, we fix variational distributions and use SGD to update parameters of NTPPs. Extensive experimental results on four synthetic datasets and three real-world datasets show the effectiveness of NTPP-MIX against state-of-the-arts.","doi":"10.24963/ijcai.2022/523","url":"https://ijcai.org/proceedings/2022/523","pdf":"https://ijcai.org/proceedings/2022/0523.pdf","markdown":"https://ijcai.org/proceedings/2022/523.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/523","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":523,"pages":{"first":3766,"last":3772},"keywords":["Machine Learning: Clustering","Machine Learning: Time-series; Data Streams"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986456","length_sec":80,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984599","length_sec":334,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bishop and Nasrabadi, 2007] Christopher M. Bishop and Nasser M. Nasrabadi. Pattern Recognition and Machine Learning. J. Electronic Imaging, 2007.","title":"Pattern Recognition and Machine Learning","authors":"Christopher M Bishop, Nasser M Nasrabadi","year":"2007","venue":"J. Electronic Imaging","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2018] Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Kristjanson Duvenaud. Neural or- dinary differential equations. In NeurIPS, 2018.","title":"Neural ordinary differential equations","authors":"Tian Qi, Chen, Yulia Rubanova, Jesse Bettencourt, David Kristjanson, Duvenaud","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2021] Ricky T. Q. Chen, Brandon Amos, and Maximilian Nickel. Neural spatio-temporal point pro- cesses. In ICLR, 2021.","title":"Neural spatio-temporal point processes","authors":"T Q Ricky, Brandon Chen, Maximilian Amos, Nickel","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Daley and David, 2007] D.J. Daley and Vere-Jones David. An Introduction to the Theory of Point Processes: Vol- ume II: General Theory and Structure. Springer Science & Business Media, 2007.","title":"An Introduction to the Theory of Point Processes: Volume II: General Theory and Structure","authors":"D J Daley, Vere-Jones David","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Du et al., 2016] Nan Du, H. Dai, Rakshit S. Trivedi, U. Upadhyay, M. Gomez-Rodriguez, and Le Song. Recur- rent marked temporal point processes: Embedding event history to vector. In KDD, 2016.","title":"Recurrent marked temporal point processes: Embedding event history to vector","authors":"Nan Du, H Dai, Rakshit S Trivedi, U Upadhyay, M Gomez-Rodriguez, Le Song","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Enguehard et al., 2020] Joseph Enguehard, Dan Busbridge, Adam James Bozson, Claire Woodcock, and Nils Y. Ham- merla. Neural temporal point processes for modelling electronic health records. In ML4H, 2020.","title":"Neural temporal point processes for modelling electronic health records","authors":"Joseph Enguehard, Dan Busbridge, Adam James Bozson, Claire Woodcock, Nils Y Hammerla","year":"2020","venue":"ML4H","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hawkes, 1971] Alan G. Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 1971.","title":"Spectra of some self-exciting and mutually exciting point processes","authors":"Alan G Hawkes","year":"1971","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hoffman et al., 2013] Matthew D. Hoffman, David M. Blei, Wang Chong, and John Paisley. Stochastic variational in- ference. JMLR, 14, 2013.","title":"Stochastic variational inference","authors":"Matthew D Hoffman, David M Blei, Wang Chong, John Paisley","year":"2013","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Iwayama et al., 2017] Koji Iwayama, Yoshito Hirata, and Kazuyuki Aihara. Definition of distance for nonlinear time series analysis of marked point process data. Physics Let- ters A, 2017.","title":"Definition of distance for nonlinear time series analysis of marked point process data","authors":"Koji Iwayama, Yoshito Hirata, Kazuyuki Aihara","year":"2017","venue":"Physics Letters A","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jia and Benson, 2019] Junteng Jia and Austin R Benson. Neural jump stochastic differential equations. In NeurIPS, 2019.","title":"Neural jump stochastic differential equations","authors":"Benson ; Jia, Junteng Jia, Austin R Benson","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2018] Shuang Li, Shuai Xiao, Shixiang Zhu, Nan Du, Yao Xie, and Le Song. Learning temporal point pro- cesses via reinforcement learning. In NeurIPS, 2018.","title":"Learning temporal point processes via reinforcement learning","authors":"Shuang Li, Shuai Xiao, Shixiang Zhu, Nan Du, Yao Xie, Le Song","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mei and Eisner, 2016] Hongyuan Mei and Jason Eisner. The neural hawkes process: A neurally self-modulating multivariate point process. In NeurIPS, 2016.","title":"The neural hawkes process: A neurally self-modulating multivariate point process","authors":"Eisner ; Mei, Hongyuan Mei, Jason Eisner","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nguyen et al., 2010] Xuan Vinh Nguyen, Julien Epps, and James Bailey. Information theoretic measures for cluster- ings comparison: Variants, properties, normalization and correction for chance. JMLR, 2010.","title":"Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance","authors":"Xuan Vinh Nguyen, Julien Epps, James Bailey","year":"2010","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Omi et al., 2019] Takahiro Omi, Naonori Ueda, and Kazuyuki Aihara. Fully neural network based model for general temporal point processes. In NeurIPS, 2019.","title":"Fully neural network based model for general temporal point processes","authors":"Takahiro Omi, Naonori Ueda, Kazuyuki Aihara","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Opper and Saad, 2001] Manfred Opper and David Saad. Advanced mean field methods: theory and practice. MIT press, 2001.","title":"Advanced mean field methods: theory and practice","authors":"Saad Opper, Manfred Opper, David Saad","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sharma et al., 2021] Karishma Sharma, Yizhou Zhang, Emilio Ferrara, and Yan Liu. Identifying coordinated ac- counts on social media through hidden influence and group behaviours. KDD, 2021.","title":"Identifying coordinated accounts on social media through hidden influence and group behaviours","authors":"Karishma Sharma, Yizhou Zhang, Emilio Ferrara, Yan Liu","year":"2021","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shchur et al., 2020] Oleksandr Shchur, Marin Biloš, and Stephan Günnemann. Intensity-free learning of temporal point processes. In ICLR, 2020.","title":"Intensity-free learning of temporal point processes","authors":"Oleksandr Shchur, Marin Biloš, Stephan Günnemann","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sutskever et al., 2013] Ilya Sutskever, James Martens, George E. Dahl, and Geoffrey E. Hinton. On the impor- tance of initialization and momentum in deep learning. In ICML, 2013.","title":"On the importance of initialization and momentum in deep learning","authors":"Ilya Sutskever, James Martens, George E Dahl, Geoffrey E Hinton","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Upadhyay et al., 2018] U. Upadhyay, Abir De, and Manuel Gomez-Rodriguez. Deep reinforcement learning of marked temporal point processes. In NeurIPS, 2018.","title":"Deep reinforcement learning of marked temporal point processes","authors":"U Upadhyay, Abir De, Manuel Gomez-Rodriguez","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wu et al., 2022] Weichang Wu, Junchi Yan, Xiaokang Yang, and Hongyuan Zha. Discovering temporal patterns for event sequence clustering via policy mixture model. TKDE, 2022.","title":"Discovering temporal patterns for event sequence clustering via policy mixture model","authors":"Weichang Wu, Junchi Yan, Xiaokang Yang, Hongyuan Zha","year":"2022","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xiao et al., 2017] Shuai Xiao, Mehrdad Farajtabar, Xiao- jing Ye, Junchi Yan, Xiaokang Yang, Le Song, and Hongyuan Zha. Wasserstein learning of deep generative point process models. In NeurIPS, 2017.","title":"Wasserstein learning of deep generative point process models","authors":"Shuai Xiao, Mehrdad Farajtabar, Xiaojing Ye, Junchi Yan, Xiaokang Yang, Le Song, Hongyuan Zha","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu and Zha, 2017] Hongteng Xu and Hongyuan Zha. A dirichlet mixture model of hawkes processes for event se- quence clustering. In NeurIPS, 2017.","title":"A dirichlet mixture model of hawkes processes for event sequence clustering","authors":"Zha Xu, Hongteng Xu, Hongyuan Zha","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xu et al., 2014] Lizhen Xu, Jason A. Duan, and Andrew Whinston. Path to purchase: A mutually exciting point process model for online advertising and conversion. Man- agement Science, 2014.","title":"Path to purchase: A mutually exciting point process model for online advertising and conversion","authors":"Lizhen Xu, Jason A Duan, Andrew Whinston","year":"2014","venue":"Management Science","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xu et al., 2016] Hongteng Xu, Mehrdad Farajtabar, and Hongyuan Zha. Learning granger causality for hawkes processes. In ICML, 2016.","title":"Learning granger causality for hawkes processes","authors":"Xu","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yan et al., 2016] Junchi Yan, Shuai Xiao, Changsheng Li, Bo Jin, Xiangfeng Wang, Bin Ke, Xiaokang Yang, and Hongyuan Zha. Modeling contagious merger and acqui- sition via point processes with a profile regression prior. In IJCAI, 2016.","title":"Modeling contagious merger and acquisition via point processes with a profile regression prior","authors":"Yan","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/382"},{"position":26,"raw":"[Yan et al., 2018] Junchi Yan, Xin Liu, Liangliang Shi, Changsheng Li, and Hongyuan Zha. Improving maxi- mum likelihood estimation of temporal point process via discriminative and adversarial learning. In IJCAI, 2018.","title":"Improving maximum likelihood estimation of temporal point process via discriminative and adversarial learning","authors":"Yan","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/409"},{"position":27,"raw":"[Zhang and Yan, 2021] Yunhao Zhang and Junchi Yan. Neu- ral relation inference for multi-dimensional temporal point processes via message passing graph. In IJCAI, 2021.","title":"Neural relation inference for multi-dimensional temporal point processes via message passing graph","authors":"Yunhao Zhang, Junchi Yan","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/469"},{"position":28,"raw":"[Zhang et al., 2020] Qiang Zhang, Aldo Lipani, Omer Kir- nap, and Emine Yilmaz. Self-attentive hawkes process. In ICML, 2020.","title":"Self-attentive hawkes process","authors":"Qiang Zhang, Aldo Lipani, Omer Kirnap, Emine Yilmaz","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou et al., 2013] K. Zhou, H. Zha, and L. Song. Learning social infectivity in sparse low-rank networks using multi- dimensional hawkes processes. In AISTATS, 2013.","title":"Learning social infectivity in sparse low-rank networks using multidimensional hawkes processes","authors":"K Zhou, H Zha, L Song","year":"2013","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zuo et al., 2020] Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao, and Hongyuan Zha. Transformer hawkes pro- cess. In ICML, 2020.","title":"Transformer hawkes process","authors":"Zuo","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}