{"type":"ScholarlyArticle","title":"Toeplitz Inverse Covariance-based Clustering of Multivariate Time Series Data","authors":[{"name":"David Hallac","slug":"david-hallac","given":"David","family":"Hallac","affiliation":"Stanford University"},{"name":"Sagar Vare","slug":"sagar-vare","given":"Sagar","family":"Vare","affiliation":"Stanford University"},{"name":"Stephen Boyd","slug":"stephen-boyd","given":"Stephen","family":"Boyd","affiliation":"Stanford University"},{"name":"Jure Leskovec","slug":"jure-leskovec","given":"Jure","family":"Leskovec","affiliation":"Stanford University"}],"abstract":"Subsequence clustering of multivariate time series is a useful tool for discovering repeated patterns in temporal data. Once these patterns have been discovered, seemingly complicated datasets can be interpreted as a temporal sequence of only a small number of states, or clusters. However, discovering these patterns is challenging because it requires simultaneous segmentation and clustering of the time series. Here we propose a new method of model-based clustering, which we call Toeplitz Inverse Covariance-based Clustering (TICC). Each cluster in the TICC method is defined by a correlation network, or Markov random field (MRF), characterizing the interdependencies between different observations in a typical subsequence of that cluster. Based on this graphical representation, TICC simultaneously segments and clusters the time series data. We solve the TICC problem through a scalable algorithm that is able to efficiently solve for tens of millions of observations. We validate our approach by comparing TICC to several state-of-the-art baselines in a series of synthetic experiments, and we then demonstrate on an automobile dataset how TICC can be used to learn interpretable clusters in real-world scenarios.","doi":"10.24963/ijcai.2018/732","url":"https://ijcai.org/proceedings/2018/732","pdf":"https://ijcai.org/proceedings/2018/0732.pdf","markdown":"https://ijcai.org/proceedings/2018/732.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/732","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Sister Conferences Best Papers","track":"Best Sister Conferences","number":732,"pages":{"first":5254,"last":5258},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Time-series;Data Streams"],"videos":[],"references":[{"position":1,"raw":"[Banerjee et al., 2008] Onureena Banerjee, Laurent El Ghaoui, and Alexandre d'Aspremont. Model se- lection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. JMLR, 2008.","title":"Alexandre d'Aspremont. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data","authors":"Banerjee","year":"2008","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Banfield and Raftery, 1993] Jeffrey D Banfield and Adrian E Raftery. Model-based Gaussian and non- Gaussian clustering. Biometrics, 1993.","title":"Model-based Gaussian and non-Gaussian clustering","authors":"D Jeffrey, Adrian E Banfield, Raftery","year":"1993","venue":"Biometrics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Begum et al., 2015] Nurjahan Begum, Liudmila Ulanova, Jun Wang, and Eamonn Keogh. Accelerating dynamic time warping clustering with a novel admissible pruning strategy. In KDD, 2015.","title":"Accelerating dynamic time warping clustering with a novel admissible pruning strategy","authors":"Nurjahan Begum, Liudmila Ulanova, Jun Wang, Eamonn Keogh","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Berndt and Clifford, 1994] Donald J Berndt and James Clif- ford. Using dynamic time warping to find patterns in time series. In AAAI Workshop on Knowledge Disovery in Databases, 1994.","title":"Using dynamic time warping to find patterns in time series","authors":"J Donald, James Berndt, Clifford","year":"1994","venue":"AAAI Workshop on Knowledge Disovery in Databases","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Boyd et al., 2011] Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. Distributed opti- mization and statistical learning via the alternating direc- tion method of multipliers. Foundations and Trends in Ma- chine Learning, 2011.","title":"Distributed optimization and statistical learning via the alternating direction method of multipliers","authors":"Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein","year":"2011","venue":"Foundations and Trends in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Brandes, 2001] Ulrik Brandes. A faster algorithm for be- tweenness centrality. Journal of Mathematical Sociology, 2001.","title":"A faster algorithm for betweenness centrality","authors":"Ulrik Brandes","year":"2001","venue":"Journal of Mathematical Sociology","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cuturi, 2011] Marco Cuturi. Fast global alignment kernels. In ICML, 2011.","title":"Fast global alignment kernels","authors":"Marco Cuturi","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dimtriadou, 2009] Evgenia Dimtriadou. Cclust: Con- vex clustering methods and clustering indexes. https://CRAN.","title":"Cclust: Convex clustering methods and clustering indexes","authors":"Evgenia Dimtriadou","year":null,"venue":null,"doi":null,"url":"https://CRAN","internal_url":null},{"position":9,"raw":"R-project.org/package=cclust, 2009.","title":"R-project","authors":null,"year":"2009","venue":null,"doi":null,"url":"org/package=cclust","internal_url":null},{"position":10,"raw":"[Fraley and Raftery, 2006] Chris Fraley and Adrian E Raftery. MCLUST version 3: an R package for normal mixture modeling and model-based clustering. Technical report, DTIC Document, 2006.","title":"MCLUST version 3: an R package for normal mixture modeling and model-based clustering","authors":"Chris Fraley, Adrian E Raftery","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Friedman et al., 2008] Jerome Friedman, Trevor Hastie, and Robert Tibshirani. Sparse inverse covariance estima- tion with the graphical lasso. Biostatistics, 2008.","title":"Sparse inverse covariance estimation with the graphical lasso","authors":"Jerome Friedman, Trevor Hastie, Robert Tibshirani","year":"2008","venue":"Biostatistics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hallac et al., 2016] David Hallac, Peter Nystrup, and Stephen Boyd. Greedy Gaussian segmentation of mul- tivariate time series. arXiv preprint arXiv:1610.07435, 2016.","title":"Greedy Gaussian segmentation of multivariate time series","authors":"David Hallac, Peter Nystrup, Stephen Boyd","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Himberg et al., 2001] Johan Himberg, Kalle Korpiaho, Heikki Mannila, Johanna Tikänmaki, and Hannu TT Toivonen. Time series segmentation for context recogni- tion in mobile devices. In ICDM, 2001.","title":"Time series segmentation for context recognition in mobile devices","authors":"Johan Himberg, Kalle Korpiaho, Heikki Mannila, Johanna Tikänmaki, Hannu Tt Toivonen","year":"2001","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Koller and Friedman, 2009] Daphne Koller and Nir Fried- man. Probabilistic Graphical Models: Principles and Techniques. MIT press, 2009.","title":"Daphne Koller and Nir Friedman","authors":"Friedman Koller","year":"2009","venue":"Probabilistic Graphical Models: Principles and Techniques","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lauritzen, 1996] Steffen L Lauritzen. Graphical models. Clarendon Press, 1996.","title":"Graphical models","authors":"Steffen L Lauritzen","year":"1996","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Miyajima et al., 2007] Chiyomi Miyajima, Yoshihiro Nishiwaki, Koji Ozawa, Toshihiro Wakita, Katsunobu Itou, Kazuya Takeda, and Fumitada Itakura. Driver modeling based on driving behavior and its evaluation in driver identification. Proceedings of the IEEE, 2007.","title":"Driver modeling based on driving behavior and its evaluation in driver identification","authors":"Miyajima","year":"2007","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mörchen et al., 2005] Fabian Mörchen, Alfred Ultsch, and Olaf Hoos. Extracting interpretable muscle activation pat- terns with time series knowledge mining. International Journal of Knowledge-based and Intelligent Engineering Systems, 2005.","title":"Extracting interpretable muscle activation patterns with time series knowledge mining","authors":"Fabian Mörchen, Alfred Ultsch, Olaf Hoos","year":"2005","venue":"International Journal of Knowledge-based and Intelligent Engineering Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Namaki et al., 2011] A Namaki, AH Shirazi, R Raei, and GR Jafari. Network analysis of a financial market based on genuine correlation and threshold method. Physica A: Stat. Mech. Apps., 2011.","title":"Network analysis of a financial market based on genuine correlation and threshold method","authors":"Namaki, R Shirazi, Raei, Jafari","year":"2011","venue":"Physica A: Stat. Mech. Apps","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Parikh and Boyd, 2014] Neal Parikh and Stephen Boyd. Proximal algorithms. Foundations and Trends in Opti- mization, 2014.","title":"Proximal algorithms","authors":"Neal Parikh, Stephen Boyd","year":"2014","venue":"Foundations and Trends in Optimization","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rue and Held, 2005] Havard Rue and Leonhard Held. Gaussian Markov Random Fields: Theory and Applica- tions. CRC Press, 2005.","title":"Gaussian Markov Random Fields: Theory and Applications","authors":"Havard Rue, Leonhard Held","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sarda, 2016] Alexis Sarda. Dtwclust. https://cran.r- project.org/web/packages/dtwclust/index.html, 2016.","title":"Dtwclust","authors":"Alexis Sarda","year":"2016","venue":null,"doi":null,"url":"https://cran.r-project.org/web/packages/dtwclust/index.html","internal_url":null},{"position":22,"raw":"[Smyth, 1997] Padhraic Smyth. Clustering sequences with hidden Markov models. NIPS, 1997.","title":"Clustering sequences with hidden Markov models","authors":"Padhraic Smyth","year":"1997","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Viterbi, 1967] Andrew Viterbi. Error bounds for convolu- tional codes and an asymptotically optimum decoding al- gorithm. IEEE Transactions on Information Theory, 1967.","title":"Error bounds for convolutional codes and an asymptotically optimum decoding algorithm","authors":"Andrew Viterbi","year":"1967","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wainwright and Jordan, 2006] Martin J Wainwright and Michael I Jordan. Log-determinant relaxation for approxi- mate inference in discrete Markov random fields. IEEE Tr. on Signal Processing, 2006.","title":"Log-determinant relaxation for approximate inference in discrete Markov random fields","authors":"J Martin, Michael I Wainwright, Jordan","year":"2006","venue":"IEEE Tr. on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wytock and Kolter, 2013] Matt Wytock and J Zico Kolter. Sparse Gaussian conditional random fields: Algorithms, theory, and application to energy forecasting. ICML, 2013.","title":"Sparse Gaussian conditional random fields: Algorithms, theory, and application to energy forecasting","authors":"Kolter Wytock, Matt Wytock, J Zico, Kolter","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yuan and Lin, 2006] Ming Yuan and Yi Lin. Model selec- tion and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B, 68:49- 67, 2006.","title":"Model selection and estimation in regression with grouped variables","authors":"Lin ; Yuan, Ming Yuan, Yi Lin","year":"2006","venue":"Journal of the Royal Statistical Society: Series B","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}