{"type":"ScholarlyArticle","title":"INITIATOR: Noise-contrastive Estimation for Marked Temporal Point Process","authors":[{"name":"Ruocheng Guo","slug":"ruocheng-guo","given":"Ruocheng","family":"Guo","affiliation":"Computer Science and Engineering , Arizona State University , USA"},{"name":"Jundong Li","slug":"jundong-li","given":"Jundong","family":"Li","affiliation":"Computer Science and Engineering , Arizona State University , USA"},{"name":"Huan Liu","slug":"huan-liu","given":"Huan","family":"Liu","affiliation":"Computer Science and Engineering , Arizona State University , USA"}],"abstract":"Copious sequential event data has consistently increased in various high-impact domains such as social media and sharing economy. When events start to take place in a sequential fashion, an important question arises: \"what type of event will happen at what time in the near future?\" To answer the question, a class of mathematical models called the marked temporal point process is often exploited as it can model the timing and properties of events seamlessly in a joint framework. Recently, various recurrent neural network (RNN) models are proposed to enhance the predictive power of mark temporal point process. However, existing marked temporal point models are fundamentally based on the Maximum Likelihood Estimation (MLE) framework for the training, and inevitably suffer from the problem resulted from the intractable likelihood function. Surprisingly, little attention has been paid to address this issue. In this work, we propose INITIATOR - a novel training framework based on noise-contrastive estimation to resolve this problem. Theoretically, we show the exists a strong connection between the proposed INITIATOR and the exact MLE. Experimentally, the efficacy of INITIATOR is demonstrated over the state-of-the-art approaches on several real-world datasets from various areas.","doi":"10.24963/ijcai.2018/303","url":"https://ijcai.org/proceedings/2018/303","pdf":"https://ijcai.org/proceedings/2018/0303.pdf","markdown":"https://ijcai.org/proceedings/2018/303.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/303","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":303,"pages":{"first":2191,"last":2197},"keywords":["Machine Learning: Time-series;Data Streams","Machine Learning: Machine Learning","Machine Learning: Probabilistic Machine Learning","Machine Learning: Data Mining"],"videos":[],"references":[{"position":1,"raw":"[Arjovsky and Bottou, 2017] Martin Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. arXiv preprint arXiv:1701.04862, 2017.","title":"Towards principled methods for training generative adversarial networks","authors":"Bottou Arjovsky, Martin Arjovsky, Léon Bottou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bromley et al., 1994] Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah. Signature verification using a\" siamese\" time delay neural network. In NIPS, pages 737-744, 1994.","title":"Signature verification using a\" siamese\" time delay neural network","authors":"Jane Bromley, Isabelle Guyon, Yann Lecun, Eduard Säckinger, Roopak Shah","year":"1994","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Du et al., 2016] Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, and Le Song. Recurrent marked temporal point processes: Embedding event history to vector. In KDD, pages 1555- 1564. ACM, 2016.","title":"Recurrent marked temporal point processes: Embedding event history to vector","authors":"Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, Le Song","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Flanders, 1973] Harley Flanders. Differentiation under the integral sign. The American Mathematical Monthly, 80(6):615-627, 1973.","title":"Harley Flanders. Differentiation under the integral sign","authors":"Flanders","year":"1973","venue":"The American Mathematical Monthly","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gao and Liu, 2015] Huiji Gao and Huan Liu. Mining hu- man mobility in location-based social networks. Synthe- sis Lectures on Data Mining and Knowledge Discovery, 7(2):1-115, 2015.","title":"Mining human mobility in location-based social networks","authors":"Liu ; ] Huiji Gao, Huan Gao, Liu","year":"2015","venue":"Synthesis Lectures on Data Mining and Knowledge Discovery","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gao et al., 2015] Shuai Gao, Jun Ma, and Zhumin Chen. Modeling and predicting retweeting dynamics on mi- croblogging platforms. In WSDM, pages 107-116. ACM, 2015.","title":"Modeling and predicting retweeting dynamics on microblogging platforms","authors":"Shuai Gao, Jun Ma, Zhumin Chen","year":"2015","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Glorot and Bengio, 2010] Xavier Glorot and Yoshua Ben- gio. Understanding the difficulty of training deep feedfor- ward neural networks. In AISTATS, pages 249-256, 2010.","title":"Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks","authors":"Bengio Glorot","year":"2010","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goodfellow, 2014] Ian J Goodfellow. On distinguishability criteria for estimating generative models. arXiv preprint arXiv:1412.6515, 2014.","title":"On distinguishability criteria for estimating generative models","authors":"Ian J Goodfellow","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guo and Shakarian, 2016] Ruocheng Guo and Paulo Shakarian. A comparison of methods for cascade prediction. In ASONAM, pages 591-598. IEEE, 2016.","title":"Ruocheng Guo and Paulo Shakarian. A comparison of methods for cascade prediction","authors":"Shakarian Guo","year":"2016","venue":"ASONAM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gutmann and Hyvärinen, 2010] Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In AISTATS, pages 297-304, 2010.","title":"Noise-contrastive estimation: A new estimation principle for unnormalized statistical models","authors":"Hyvärinen Gutmann, Michael Gutmann, Aapo Hyvärinen","year":"2010","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hawkes, 1971] Alan G Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83-90, 1971.","title":"Spectra of some self-exciting and mutually exciting point processes","authors":"Alan G Hawkes","year":"1971","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kingma and Ba, 2014] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"Diederik Kingma, Jimmy Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingman, 1993] John Frank Charles Kingman. Poisson pro- cesses. Wiley Online Library, 1993.","title":"Poisson processes","authors":"John Frank, Charles Kingman","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lian et al., 2015] Wenzhao Lian, Ricardo Henao, Vinayak Rao, Joseph Lucas, and Lawrence Carin. A multitask point process predictive model. In ICML, pages 2030-2038, 2015.","title":"A multitask point process predictive model","authors":"Lian","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2017] Xin Liu, Junchi Yan, Shuai Xiao, Xi- angfeng Wang, Hongyuan Zha, and Stephen M Chu. On predictive patent valuation: Forecasting patent citations and their types. In AAAI, pages 1438-1444, 2017.","title":"On predictive patent valuation: Forecasting patent citations and their types","authors":"Xin Liu, Junchi Yan, Shuai Xiao, Xiangfeng Wang, Hongyuan Zha, Stephen M Chu","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mei and Eisner, 2016] Hongyuan Mei and Jason Eis- ner. The neural hawkes process: A neurally self- modulating multivariate point process. arXiv preprint arXiv:1612.09328, 2016.","title":"The neural hawkes process: A neurally selfmodulating multivariate point process","authors":"Eisner ; Mei, Hongyuan Mei, Jason Eisner","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed rep- resentations of words and phrases and their composition- ality. In NIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mnih and Teh, 2012] Andriy Mnih and Yee Whye Teh. A fast and simple algorithm for training neural probabilistic language models. In ICML, pages 419-426. Omnipress, 2012.","title":"A fast and simple algorithm for training neural probabilistic language models","authors":"Andriy Mnih, Yee Whye Teh","year":"2012","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rasmussen, 2011] Jakob Gulddahl Rasmussen. Temporal point processes the conditional intensity function. Lecture Notes, Jan, 2011.","title":"Temporal point processes the conditional intensity function","authors":"Jakob Gulddahl, Rasmussen","year":"2011","venue":"Lecture Notes","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shen et al., 2014] Hua-Wei Shen, Dashun Wang, Chaoming Song, and Albert-László Barabási. Modeling and predict- ing popularity dynamics via reinforced poisson processes. In AAAI, volume 14, pages 291-297, 2014.","title":"Modeling and predicting popularity dynamics via reinforced poisson processes","authors":"Hua-Wei Shen, Dashun Wang, Chaoming Song, Albert-László Barabási","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Snyder and Miller, 2012] Donald L Snyder and Michael I Miller. Random point processes in time and space. Springer Science & Business Media, 2012.","title":"Random point processes in time and space","authors":"L Donald, Michael I Snyder, Miller","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tabibian et al., 2017] Behzad Tabibian, Isabel Valera, Mehrdad Farajtabar, Le Song, Bernhard Schölkopf, and Manuel Gomez-Rodriguez. Distilling information reliability and source trustworthiness from digital traces. In WWW, pages 847-855, 2017.","title":"Distilling information reliability and source trustworthiness from digital traces","authors":"Behzad Tabibian, Isabel Valera, Mehrdad Farajtabar, Le Song, Bernhard Schölkopf, Manuel Gomez-Rodriguez","year":"2017","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Valera and Gomez-Rodriguez, 2015] Isabel Valera and Manuel Gomez-Rodriguez. Modeling adoption and usage of competing products. In ICDM, pages 409-418. IEEE, 2015.","title":"Isabel Valera and Manuel Gomez-Rodriguez. Modeling adoption and usage of competing products","authors":"Gomez-Rodriguez Valera","year":"2015","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xiao et al., 2017] Shuai Xiao, Junchi Yan, Xiaokang Yang, Hongyuan Zha, and Stephen M Chu. Modeling the in- tensity function of point process via recurrent neural net- works. In AAAI, pages 1597-1603, 2017.","title":"Modeling the intensity function of point process via recurrent neural networks","authors":"Shuai Xiao, Junchi Yan, Xiaokang Yang, Hongyuan Zha, Stephen M Chu","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhao et al., 2015] Qingyuan Zhao, Murat A Erdogdu, Hera Y He, Anand Rajaraman, and Jure Leskovec. Seis- mic: A self-exciting point process model for predicting tweet popularity. In KDD, pages 1513-1522. ACM, 2015.","title":"Seismic: A self-exciting point process model for predicting tweet popularity","authors":"Qingyuan Zhao, A Murat, Erdogdu, Y Hera, Anand He, Jure Rajaraman, Leskovec","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}