{"type":"ScholarlyArticle","title":"Similarity Preserving Representation Learning for Time Series Clustering","authors":[{"name":"Qi Lei","slug":"qi-lei","given":"Qi","family":"Lei","affiliation":"University of Texas at Austin"},{"name":"Jinfeng Yi","slug":"jinfeng-yi","given":"Jinfeng","family":"Yi","affiliation":"JD AI Research"},{"name":"Roman Vaculin","slug":"roman-vaculin","given":"Roman","family":"Vaculin","affiliation":"IBM Research"},{"name":"Lingfei Wu","slug":"lingfei-wu","given":"Lingfei","family":"Wu","affiliation":"IBM Research"},{"name":"Inderjit S. Dhillon","slug":"inderjit-s-dhillon","given":"Inderjit S.","family":"Dhillon","affiliation":"Amazon; University of Texas at Austin"}],"abstract":"A considerable amount of clustering algorithms take instance-feature matrices as their inputs. As such, they cannot directly analyze time series data due to its temporal nature, usually unequal lengths, and complex properties. This is a great pity since many of these algorithms are effective, robust, efficient, and easy to use. In this paper, we bridge this gap by proposing an efficient representation learning framework that is able to convert a set of time series with various lengths to an instance-feature matrix. In particular, we guarantee that the pairwise similarities between time series are well preserved after the transformation , thus the learned feature representation is particularly suitable for the time series clustering task. Given a set of $n$ time series, we first construct an $n\\times n$ partially-observed similarity matrix by randomly sampling $\\mathcal{O}(n \\log n)$ pairs of time series and computing their pairwise similarities. We then propose an efficient algorithm that solves a non-convex and NP-hard problem to learn new features based on the partially-observed similarity matrix. By conducting extensive empirical studies, we demonstrate that the proposed framework is much more effective, efficient, and flexible compared to other state-of-the-art clustering methods.","doi":"10.24963/ijcai.2019/394","url":"https://ijcai.org/proceedings/2019/394","pdf":"https://ijcai.org/proceedings/2019/0394.pdf","markdown":"https://ijcai.org/proceedings/2019/394.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/394","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":394,"pages":{"first":2845,"last":2851},"keywords":["Machine Learning: Data Mining","Natural Language Processing: Embeddings","Machine Learning: Clustering"],"videos":[],"references":[{"position":1,"raw":"[Bagnall et al., 2017] Anthony Bagnall, Jason Lines, Aaron Bostrom, James Large, and Eamonn J. Keogh. The great time series classification bake off: a review and experimen- tal evaluation of recent algorithmic advances. Data Min. Knowl. Discov., 31(3):606-660, 2017.","title":"The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances","authors":"Anthony Bagnall, Jason Lines, Aaron Bostrom, James Large, Eamonn J Keogh","year":"2017","venue":"Data Min. Knowl. Discov","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cardano and Witmer, 1993] G. Cardano and T R. Witmer. Ars magna or the rules of algebra. 1993.","title":"Ars magna or the rules of algebra","authors":"G Cardano, T R Witmer","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Celebi et al., 2013] S. Celebi, A. Aydin, T. Temiz, and T. Arici. Gesture recognition using skeleton data with weighted dynamic time warping. In VISAPP (1), pages 620-625, 2013.","title":"Gesture recognition using skeleton data with weighted dynamic time warping","authors":"S Celebi, A Aydin, T Temiz, T Arici","year":"2013","venue":"VISAPP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chan and Fu, 1999] Kin-Pong Chan and Ada Wai-Chee Fu. Efficient time series matching by wavelets. In ICDE, pages 126-133. IEEE, 1999.","title":"Efficient time series matching by wavelets","authors":"Chan, Fu ; Kin-Pong Chan, Ada Wai, -Chee Fu","year":"1999","venue":"ICDE","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2015] Y. Chen, E. Keogh, B. Hu, N. Begum, A. Bagnall, A. Mueen, and G. Batista. The UCR time series classification archive, July 2015. www.cs.ucr.edu/ ∼ eamonn/time series data/.","title":"The UCR time series classification archive","authors":"Y Chen, E Keogh, B Hu, N Begum, A Bagnall, A Mueen, G Batista","year":"2015","venue":null,"doi":null,"url":"www.cs.ucr.edu/∼eamonn/timeseriesdata/","internal_url":null},{"position":6,"raw":"[Deng et al., 2013] H. Deng, G. Runger, E. Tuv, and M. Vladimir. A time series forest for classification and feature extraction. Information Sciences, 239:142-153, 2013.","title":"A time series forest for classification and feature extraction","authors":"H Deng, G Runger, E Tuv, M Vladimir","year":"2013","venue":"Information Sciences","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Efrat et al., 2007] A. Efrat, Q. Fan, and S. Venkatasubrama- nian. Curve matching, time warping, and light fields: New algorithms for computing similarity between curves. Jour- nal of Mathematical Imaging and Vision, 27(3):203-216, 2007.","title":"Curve matching, time warping, and light fields: New algorithms for computing similarity between curves","authors":"A Efrat, Q Fan, S","year":"2007","venue":"Journal of Mathematical Imaging and Vision","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Elman, 1990] Jeffrey L Elman. Finding structure in time. Cognitive science, 14(2):179-211, 1990.","title":"Finding structure in time","authors":"Jeffrey L Elman","year":"1990","venue":"Cognitive science","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Erdős et al., 2009] L. Erdős, B. Schlein, and H. Yau. Local semicircle law and complete delocalization for wigner ran- dom matrices. Communications in Mathematical Physics, 287(2):641-655, 2009.","title":"Local semicircle law and complete delocalization for wigner random matrices","authors":"L Erdős, B Schlein, H Yau","year":"2009","venue":"Communications in Mathematical Physics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Faloutsos et al., 1994] C. Faloutsos, M. Ranganathan, and Y. Manolopoulos. Fast subsequence matching in time-series databases, volume 23. ACM, 1994.","title":"Fast subsequence matching in time-series databases","authors":"C Faloutsos, M Ranganathan, Y Manolopoulos","year":"1994","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fulcher and Jones, 2014] Ben D Fulcher and Nick S Jones. Highly comparative feature-based time-series classification. TKDE, 26(12):3026-3037, 2014.","title":"Highly comparative feature-based time-series classification","authors":"D Ben, Nick S Fulcher, Jones","year":"2014","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ge et al., 2016] R. Ge, J. Lee, and T. Ma. Matrix completion has no spurious local minimum. In NIPS, pages 2973-2981, 2016.","title":"Matrix completion has no spurious local minimum","authors":"R Ge, J Lee, T Ma","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hayashi et al., 2005] Akira Hayashi, Yuko Mizuhara, and Nobuo Suematsu. Embedding time series data for classifi- cation. In MLDM, pages 356-365. Springer, 2005.","title":"Embedding time series data for classification","authors":"Hayashi","year":"2005","venue":"MLDM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hills et al., 2014] Jon Hills, Jason Lines, Edgaras Baranauskas, James Mapp, and Anthony Bagnall. Classification of time series by shapelet transformation. Data Min. Knowl. Discov., 28(4):851-881, 2014.","title":"Classification of time series by shapelet transformation","authors":"Hills","year":"2014","venue":"Data Min. Knowl. Discov","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kate, 2016] Rohit J. Kate. Using dynamic time warping distances as features for improved time series classification. Data Min. Knowl. Discov., 30(2):283-312, 2016.","title":"Using dynamic time warping distances as features for improved time series classification","authors":"J Rohit, Kate","year":"2016","venue":"Data Min. Knowl. Discov","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Keogh et al., 2001] E. Keogh, K. Chakrabarti, M. Pazzani, and S. Mehrotra. Dimensionality reduction for fast similar- ity search in large time series databases. Knowledge and information Systems, 3(3):263-286, 2001.","title":"Dimensionality reduction for fast similarity search in large time series databases","authors":"E Keogh, K Chakrabarti, M Pazzani, S Mehrotra","year":"2001","venue":"Knowledge and information Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Korn et al., 1997] F. Korn, H. V Jagadish, and C. Faloutsos. Efficiently supporting ad hoc queries in large datasets of time sequences. ACM SIGMOD Record, 26(2):289-300, 1997.","title":"Efficiently supporting ad hoc queries in large datasets of time sequences","authors":"F Korn, H V Jagadish, C Faloutsos","year":"1997","venue":"ACM SIGMOD Record","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lafond, 2015] Jean Lafond. Low rank matrix completion with exponential family noise. In Conference on Learning Theory, pages 1224-1243, 2015.","title":"Low rank matrix completion with exponential family noise","authors":"Jean Lafond","year":"2015","venue":"Conference on Learning Theory","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Längkvist et al., 2014] M. Längkvist, L. Karlsson, and A. Loutfi. A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recogni- tion Letters, 42:11-24, 2014.","title":"A review of unsupervised feature learning and deep learning for time-series modeling","authors":"M Längkvist, L Karlsson, A Loutfi","year":"2014","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lei et al., 2017] Qi Lei, Jinfeng Yi, Roman Vaculin, Lingfei Wu, and Inderjit S Dhillon. Similarity preserving repre- sentation learning for time series analysis. arXiv preprint arXiv:1702.03584, 2017.","title":"Similarity preserving representation learning for time series analysis","authors":"Lei","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Li and Prakash, 2011] Lei Li and B Aditya Prakash. Time series clustering: Complex is simpler! 2011.","title":"Time series clustering: Complex is simpler!","authors":"Lei Li, Aditya Prakash","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lin et al., 2007] J. Lin, E. Keogh, L. Wei, and S. Lonardi. Experiencing sax: a novel symbolic representation of time series. Data Mining and Knowledge Discovery, 15(2):107- 144, 2007.","title":"Experiencing sax: a novel symbolic representation of time series","authors":"J Lin, E Keogh, L Wei, S Lonardi","year":"2007","venue":"Data Mining and Knowledge Discovery","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Marčenko and Pastur, 1967] V. Marčenko and L. Pastur. Dis- tribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik, 1(4):457, 1967.","title":"Distribution of eigenvalues for some sets of random matrices","authors":"V Marčenko, L Pastur","year":"1967","venue":"Mathematics of the USSR-Sbornik","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Muda et al., 2010] L. Muda, M. Begam, and I Elamvazuthi. Voice recognition algorithms using mel frequency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques. arXiv preprint arXiv:1003.4083, 2010.","title":"Voice recognition algorithms using mel frequency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques","authors":"L Muda, M Begam, Elamvazuthi","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Nanopoulos et al., 2001] A. Nanopoulos, R. Alcock, and Y. Manolopoulos. Feature-based classification of time- series data. International Journal of Computer Research, 10(3):49-61, 2001.","title":"Feature-based classification of timeseries data","authors":"A Nanopoulos, R Alcock, Y Manolopoulos","year":"2001","venue":"International Journal of Computer Research","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Niennattrakul and Ratanamahatana, 2007] V. Niennattrakul and C. Ratanamahatana. Inaccuracies of shape averaging method using dynamic time warping for time series data. In International conference on computational science, pages 513-520. Springer, 2007.","title":"Inaccuracies of shape averaging method using dynamic time warping for time series data","authors":"V Niennattrakul, C Ratanamahatana","year":"2007","venue":"International conference on computational science","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Paparrizos and Gravano, 2015] J. Paparrizos and L. Gravano. k-Shape: Efficient and accurate clustering of time series. In SIGMOD, pages 1855-1870. ACM, 2015.","title":"k-Shape: Efficient and accurate clustering of time series","authors":"J Paparrizos, L Gravano","year":"2015","venue":"SIGMOD","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sempena et al., 2011] S. Sempena, N. Maulidevi, and P. Aryan. Human action recognition using dynamic time warping. In International Conference on Electrical Engi- neering and Informatics, pages 1-5. IEEE, 2011.","title":"Human action recognition using dynamic time warping","authors":"S Sempena, N Maulidevi, P Aryan","year":"2011","venue":"International Conference on Electrical Engineering and Informatics","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Stefan et al., 2013] Alexandra Stefan, Vassilis Athitsos, and Gautam Das. The move-split-merge metric for time series. IEEE transactions on Knowledge and Data Engineering, 25(6):1425-1438, 2013.","title":"The move-split-merge metric for time series","authors":"Alexandra Stefan, Athitsos Vassilis, Gautam Das","year":"2013","venue":"IEEE transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"J. Stock and M. Watson. Implica- tions of dynamic factor models for var analysis. Technical report, National Bureau of Economic Research, 2005.","title":"Implications of dynamic factor models for var analysis","authors":"J Stock, M Watson","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sun and Luo, 2015] Ruoyu Sun and Zhi-Quan Luo. Guar- anteed matrix completion via nonconvex factorization. In FOCS, pages 270-289. IEEE, 2015.","title":"Ruoyu Sun and Zhi-Quan Luo. Guaranteed matrix completion via nonconvex factorization","authors":"Luo Sun","year":"2015","venue":"FOCS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Vandaele et al., 2015] Arnaud Vandaele, Nicolas Gillis, Qi Lei, Kai Zhong, and Inderjit S. Dhillon. Coordinate descent methods for symmetric nonnegative matrix factor- ization. CoRR, abs/1509.01404, 2015.","title":"Coordinate descent methods for symmetric nonnegative matrix factorization","authors":"Arnaud Vandaele, Nicolas Gillis, Qi Lei, Kai Zhong, Inderjit S Dhillon","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang et al., 2006] X. Wang, K. Smith, and R. Hyndman. Characteristic-based clustering for time series data. DMKD, 13(3):335-364, 2006.","title":"Characteristic-based clustering for time series data","authors":"Wang","year":"2006","venue":"DMKD","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Yang and Wu, 2006] Qiang Yang and Xindong Wu. 10 chal- lenging problems in data mining research. International Journal of Information Technology & Decision Making, 5(04):597-604, 2006.","title":"10 challenging problems in data mining research","authors":"Qiang Yang, Xindong Wu","year":"2006","venue":"International Journal of Information Technology & Decision Making","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}