{"type":"ScholarlyArticle","title":"Fast Recursive Low-rank Tensor Learning for Regression","authors":[{"name":"Ming Hou","slug":"ming-hou","given":"Ming","family":"Hou","affiliation":"Department of Computer Science and Software Engineering , Laval University , Quebec , Canada"},{"name":"Brahim Chaib-draa","slug":"brahim-chaib-draa","given":"Brahim","family":"Chaib-draa","affiliation":"Department of Computer Science and Software Engineering , Laval University , Quebec , Canada"}],"abstract":"In this work, we develop a fast sequential low-rank tensor regression framework, namely recursive higher-order partial least squares (RHOPLS). It addresses the great challenges posed by the limited storage space and fast processing time required by dynamic environments when dealing with large-scale high-speed general tensor sequences. Smartly integrating a low-rank modification strategy of Tucker into a PLS-based framework, we efficiently update the regression coefficients by effectively merging the new data into the previous low-rank approximation of the model at a small-scale factor (feature) level instead of the large raw data (observation) level. Unlike batch models, which require accessing the entire data, RHOPLS conducts a blockwise recursive calculation scheme and thus only a small set of factors is needed to be stored. Our approach is orders of magnitude faster than all other methods while maintaining a highly comparable predictability with the cutting-edge batch methods, as verified on challenging real-life tasks.","doi":"10.24963/ijcai.2017/257","url":"https://ijcai.org/proceedings/2017/257","pdf":"https://ijcai.org/proceedings/2017/0257.pdf","markdown":"https://ijcai.org/proceedings/2017/257.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/257","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":257,"pages":{"first":1851,"last":1857},"keywords":["Machine Learning: Online Learning","Machine Learning: Structured Learning","Machine Learning: Time-series/Data Streams","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Bahadori et al., 2014] Mohammad Taha Bahadori, Qi Rose Yu, and Yan Liu. Fast multivariate spatio-temporal anal- ysis via low rank tensor learning. In Advance in Neural Information Processing Systems, pages 3491-3499, 2014.","title":"Fast multivariate spatio-temporal analysis via low rank tensor learning","authors":"Mohammad Taha Bahadori, Qi Rose Yu, Yan Liu","year":"2014","venue":"Advance in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bro, 1996] Rasmus Bro. Multiway calidration. multilinear PLS. Journal of Chemometrics, 10:47-61, 1996.","title":"Multiway calidration. multilinear PLS","authors":"Rasmus Bro","year":"1996","venue":"Journal of Chemometrics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chao et al., 2010] Zenas C Chao, Yasuo Nagasaka, and Naotaka Fujii. Long-term asynchronous decoding of ar- m motion using electrocorticographic signals in monkeys. Frontiers in Neuroengineering, 3, 2010.","title":"Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys","authors":"C Zenas, Yasuo Chao, Naotaka Nagasaka, Fujii","year":"2010","venue":"Frontiers in Neuroengineering","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cichocki et al., 2009] Andrzej Cichocki, Rafal Zdunek, Anh Huy Phan, and Shun-ichi Amari. Nonnegative ma- trix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separation. John Wiley and Sons, 2009.","title":"Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separation","authors":"Andrzej Cichocki, Rafal Zdunek, Anh Huy Phan, Shun-Ichi Amari","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[De Lathauwer et al., 2000a] Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4):1253-1278, 2000.","title":"A multilinear singular value decomposition","authors":"De Lathauwer","year":"2000","venue":"SIAM Journal on Matrix Analysis and Applications","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[De Lathauwer et al., 2000b] Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. On the best rank-1 and rank-(r 1, r 2,..., rn) approximation of higher-order ten- sors. SIAM Journal on Matrix Analysis and Applications, 21(4):1324-1342, 2000.","title":"On the best rank-1 and rank-(r 1, r 2,..., rn) approximation of higher-order tensors","authors":"De Lathauwer","year":"2000","venue":"SIAM Journal on Matrix Analysis and Applications","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Eliseyev and Aksenova, 2013] Andrey Eliseyev and Tetiana Aksenova. Recursive n-way partial least squares for brain- computer interface. PloS One, 8(7):e69962, 2013.","title":"Recursive n-way partial least squares for braincomputer interface","authors":"Andrey Eliseyev, Tetiana Aksenova","year":"2013","venue":"PloS One","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Harshman, 1970] Richard A Harshman. Foundations of the parafac procedure: Models and conditions for an\" explana- tory\" multi-modal factor analysis. 1970.","title":"Foundations of the parafac procedure: Models and conditions for an\" explanatory\" multi-modal factor analysis","authors":"Richard A Harshman","year":"1970","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hou and Chaib-draa, 2016] Ming Hou and Brahim Chaib- draa. Online incremental higher-order partial least squares regression for fast reconstruction of motion trajectories from tensor streams. In International Conference on A- coustics, Speech, and Signal Processing. IEEE, 2016.","title":"Online incremental higher-order partial least squares regression for fast reconstruction of motion trajectories from tensor streams","authors":"Chaib-Draa ; Hou, Ming Hou, Brahim Chaibdraa","year":"2016","venue":"International Conference on Acoustics, Speech, and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kim et al., 2005] Hyunsoo Kim, Jeff X Zhou, Herbert C Morse III, and Haesun Park. A three-stage framework for gene expression data analysis by L1-norm support vector regression. International Journal of Bioinformatics Re- search and Applications, 1(1):51-62, 2005.","title":"A three-stage framework for gene expression data analysis by L1-norm support vector regression","authors":"Hyunsoo Kim, Jeff X Zhou, Herbert C Morse, Iii, Haesun Park","year":"2005","venue":"International Journal of Bioinformatics Research and Applications","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Luo et al., 2015] Lijia Luo, Shiyi Bao, and Zengliang Gao. Quality prediction based on hopls-cp for batch process- es. Chemometrics and Intelligent Laboratory Systems, 143:28-39, 2015.","title":"Quality prediction based on hopls-cp for batch processes","authors":"Lijia Luo, Shiyi Bao, Zengliang Gao","year":"2015","venue":"Chemometrics and Intelligent Laboratory Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[O'Hara, 2010] Michael J O'Hara. On low-rank updates to the singular value and tucker decompositions. In SIAM International Conference on Data Mining, pages 713-719.","title":"On low-rank updates to the singular value and tucker decompositions","authors":"J O' Michael, Hara","year":null,"venue":"SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Sun et al., 2008] Jimeng Sun, Dacheng Tao, Spiros Pa- padimitriou, Philip S Yu, and Christos Faloutsos. Incre- mental tensor analysis: Theory and applications. Transac- tions on Knowledge Discovery from Data, 2(3):11, 2008.","title":"Incremental tensor analysis: Theory and applications","authors":"Sun","year":"2008","venue":"Transactions on Knowledge Discovery from Data","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Van Der Aa et al., 2011] NP Van Der Aa, X Luo, GJ Gieze- man, RT Tan, and RC Veltkamp. Utrecht multi-person motion (UMPM) benchmark. Technical report, Citeseer, 2011.","title":"Utrecht multi-person motion (UMPM) benchmark","authors":"Aa Van Der","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Yu et al., 2015] Rose Yu, Dehua Cheng, and Yan Liu. Ac- celerated online low-rank tensor learning for multivariate spatio-temporal streams. In International Conference on Machine Learning, pages 238-247, 2015.","title":"Accelerated online low-rank tensor learning for multivariate spatio-temporal streams","authors":"Rose Yu, Dehua Cheng, Yan Liu","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Zhao et al., 2011] Qibin Zhao, Cesar F Caiafa, Danilo P Mandic, Liqing Zhang, Tonio Ball, Andreas Schulze- Bonhage, and Andrzej Cichocki. Multilinear subspace re- gression: An orthogonal tensor decomposition approach. In Advance in Neural Information Processing Systems, volume 2011, pages 1269-1277, 2011.","title":"Multilinear subspace regression: An orthogonal tensor decomposition approach","authors":"Zhao","year":"2011","venue":"Advance in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Zhao et al., 2013] Qibin Zhao, Cesar F Caiafa, Danilo P Mandic, Zenas C Chao, Yasuo Nagasaka, Naotaka Fujii, Liqing Zhang, and Andrzej Cichocki. Higher order partial least squares (HOPLS): a generalized multilinear regres- sion method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(7):1660-1673, 2013.","title":"Higher order partial least squares (HOPLS): a generalized multilinear regression method","authors":"Zhao","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zhao et al., 2014] Qibin Zhao, Guoxu Zhou, Liqing Zhang, and Andrzej Cichocki. Tensor-variate gaussian processes regression and its application to video surveillance. In In- ternational Conference on Acoustics, Speech, and Signal Processing, pages 1265-1269. IEEE, 2014.","title":"Tensor-variate gaussian processes regression and its application to video surveillance","authors":"Zhao","year":"2014","venue":"International Conference on Acoustics, Speech, and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhou et al., 2013] Hua Zhou, Lexin Li, and Hongtu Zhu. Tensor regression with applications in neuroimaging data analysis. Journal of the American Statistical Association, 108(502):540-552, 2013.","title":"Tensor regression with applications in neuroimaging data analysis","authors":"Hua Zhou, Lexin Li, Hongtu Zhu","year":"2013","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}