{"type":"ScholarlyArticle","title":"Survey on Feature Transformation Techniques for Data Streams","authors":[{"name":"Maroua Bahri","slug":"maroua-bahri","given":"Maroua","family":"Bahri","affiliation":"LTCI , Télécom Paris , IP-Paris , Palaiseau , France"},{"name":"Albert Bifet","slug":"albert-bifet","given":"Albert","family":"Bifet","affiliation":"LTCI , Télécom Paris , IP-Paris , Palaiseau , France; University of Waikato , Hamilton , New Zealand"},{"name":"Silviu Maniu","slug":"silviu-maniu","given":"Silviu","family":"Maniu","affiliation":"Université Paris-Saclay , LRI , CNRS , Orsay , France; ENS DI , ENS , Université PSL , Paris , France; Inria , Paris , France"},{"name":"Heitor Murilo Gomes","slug":"heitor-murilo-gomes","given":"Heitor Murilo","family":"Gomes","affiliation":"University of Waikato , Hamilton , New Zealand"}],"abstract":"Mining high-dimensional data streams poses a fundamental challenge to machine learning as the presence of high numbers of attributes can remarkably degrade any mining task's performance. In the past several years, dimension reduction (DR) approaches have been successfully applied for different purposes (e.g., visualization). Due to their high-computational costs and numerous passes over large data, these approaches pose a hindrance when processing infinite data streams that are potentially high-dimensional. The latter increases the resource-usage of algorithms that could suffer from the curse of dimensionality. To cope with these issues, some techniques for incremental DR have been proposed. In this paper, we provide a survey on reduction approaches designed to handle data streams and highlight the key benefits of using these approaches for stream mining algorithms.","doi":"10.24963/ijcai.2020/668","url":"https://ijcai.org/proceedings/2020/668","pdf":"https://ijcai.org/proceedings/2020/0668.pdf","markdown":"https://ijcai.org/proceedings/2020/668.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/668","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Survey track","track":"Survey track","number":668,"pages":{"first":4796,"last":4802},"keywords":["Machine Learning: general"],"videos":[],"references":[{"position":1,"raw":"[Achlioptas, 2003] Dimitris Achlioptas. Database-friendly random projections: Johnson-lindenstrauss with binary coins. JCSS, 66(4):671-687, 2003.","title":"Database-friendly random projections: Johnson-lindenstrauss with binary coins","authors":"Dimitris Achlioptas","year":"2003","venue":"JCSS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Agarwal et al., 2010] Arvind Agarwal, Jeff M Phillips, Hal Daumé III, and Suresh Venkatasubramanian. Incremental multidimensional scaling. In LW, 2010.","title":"Incremental multidimensional scaling","authors":"Arvind Agarwal, Jeff M Phillips, Hal Daumé, Iii, Suresh Venkatasubramanian","year":"2010","venue":"LW","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Aggarwal and Philip, 2007] Charu Aggarwal and S Yu Philip. A survey of synopsis construction in data streams. In Data Streams, pages 169-207. Springer, 2007.","title":"A survey of synopsis construction in data streams","authors":"Charu Aggarwal, S Yu, Philip","year":"2007","venue":"Data Streams","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[AlNuaimi et al., 2019] Noura AlNuaimi, Moham- mad Mehedy Masud, Mohamed Adel Serhani, and Nazar Zaki. Streaming feature selection algorithms for big data: A survey. ACI, 2019.","title":"Streaming feature selection algorithms for big data: A survey","authors":"Noura Alnuaimi, Mohammad Mehedy Masud, Mohamed Adel Serhani, Nazar Zaki","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Arjoune et al., 2018] Youness Arjoune, Naima Kaabouch, Hassan El Ghazi, and Ahmed Tamtaoui. A performance comparison of measurement matrices in compressive sens- ing. IJCS, 31(10):e3576, 2018.","title":"A performance comparison of measurement matrices in compressive sensing","authors":"Youness Arjoune, Naima Kaabouch, Hassan El Ghazi, Ahmed Tamtaoui","year":"2018","venue":"IJCS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Artac et al., 2002] Matej Artac, Matjaz Jogan, and Ales Leonardis. Incremental pca for on-line visual learning and recognition. In Object Recognition, pages 781-784, 2002.","title":"Incremental pca for on-line visual learning and recognition","authors":"Matej Artac, Matjaz Jogan, Ales Leonardis","year":"2002","venue":"Object Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bahri et al., 2018] Maroua Bahri, Silviu Maniu, and Albert Bifet. Sketch-based naive bayes algorithms for evolving data streams. In ICBD, pages 604-613. IEEE, 2018.","title":"Sketch-based naive bayes algorithms for evolving data streams","authors":"Bahri","year":"2018","venue":"ICBD","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Bahri et al., 2020] Maroua Bahri, Bernhard Pfahringer, Al- bert Bifet, and Silviu Maniu. Efficient batch-incremental classification using umap for evolving data streams. In IDA, pages 40-53, 2020.","title":"Efficient batch-incremental classification using umap for evolving data streams","authors":"Bahri","year":"2020","venue":"IDA","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Barddal et al., 2017] Jean Paul Barddal, Heitor Murilo Gomes, Fabrício Enembreck, and Bernhard Pfahringer. A survey on feature drift adaptation: Definition, benchmark, challenges and future directions. JSS, 127:278-294, 2017.","title":"A survey on feature drift adaptation: Definition, benchmark, challenges and future directions","authors":"Barddal","year":"2017","venue":"JSS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Bellman, 2015] Richard E Bellman. Adaptive control pro- cesses: a guided tour. Princeton university press, 2015.","title":"Adaptive control processes: a guided tour","authors":"Richard E Bellman","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Cardot and Degras, 2018] Hervé Cardot and David Degras. Online principal component analysis in high dimension: which algorithm to choose? ISR, 86(1):29-50, 2018.","title":"Online principal component analysis in high dimension: which algorithm to choose?","authors":"Degras Cardot, Hervé Cardot, David Degras","year":"2018","venue":"ISR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Datar et al., 2004] Mayur Datar, Nicole Immorlica, Piotr Indyk, and Vahab S Mirrokni. Locality-sensitive hashing scheme based on p-stable distributions. In SoCG, 2004.","title":"Locality-sensitive hashing scheme based on p-stable distributions","authors":"Mayur Datar, Nicole Immorlica, Piotr Indyk, Vahab, Mirrokni","year":"2004","venue":"SoCG","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Dong and Japkowicz, 2016] Yue Dong and Nathalie Jap- kowicz. Threaded ensembles of supervised and unsuper- vised nns for stream learning. In CAIAC, 2016.","title":"Threaded ensembles of supervised and unsupervised nns for stream learning","authors":"Yue Dong, Nathalie Japkowicz","year":"2016","venue":"CAIAC","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Donoho, 2006] David L Donoho. Compressed sensing. TIT, 52(4):1289-1306, 2006.","title":"Compressed sensing","authors":"L David, Donoho","year":"2006","venue":"TIT","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Feng et al., 2009] Wu Feng, Zhong Yan, Li Ai-ping, and Wu Quan-yuan. Online classification algorithm for data streams based on fast iterative kernel pca. In ICNC, 2009.","title":"Online classification algorithm for data streams based on fast iterative kernel pca","authors":"Wu Feng, Zhong Yan, Li Ai-Ping, Wu Quan-Yuan","year":"2009","venue":"ICNC","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Freris et al., 2013] Nikolaos M Freris, Orhan Oc ¸al, and Martin Vetterli. Compressed sensing of streaming data. In Allerton, pages 1242-1249. IEEE, 2013.","title":"Compressed sensing of streaming data","authors":"Orhan Nikolaos M Freris, Martin Oc, Vetterli","year":"2013","venue":"Allerton","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gama et al., 2009] João Gama, Raquel Sebastião, and Pe- dro Pereira Rodrigues. Issues in evaluation of stream learning algorithms. In SIGKDD. ACM, 2009.","title":"Issues in evaluation of stream learning algorithms","authors":"João Gama, Raquel Sebastião, Pedro Pereira Rodrigues","year":"2009","venue":"SIGKDD. ACM","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Günter et al., 2007] Simon Günter, Nicol N Schraudolph, and SVN Vishwanathan. Fast iterative kernel principal component analysis. JMLR, 8(8):1893-1918, 2007.","title":"Fast iterative kernel principal component analysis","authors":"Simon Günter, Nicol N Schraudolph, Vishwanathan","year":"2007","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Haque et al., 2016] Ahsanul Haque, Latifur Khan, Michael Baron, Bhavani Thuraisingham, and Charu Aggarwal. Ef- ficient handling of concept drift and concept evolution over stream data. In ICDE, pages 481-492. IEEE, 2016.","title":"Efficient handling of concept drift and concept evolution over stream data","authors":"Haque","year":"2016","venue":"ICDE","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Hu et al., 2018] Xuegang Hu, Peng Zhou, Peipei Li, Jing Wang, and Xindong Wu. A survey on online feature selec- tion with streaming features. FCS, 12(3):479-493, 2018.","title":"A survey on online feature selection with streaming features","authors":"Xuegang Hu, Peng Zhou, Peipei Li, Jing Wang, Xindong Wu","year":"2018","venue":"FCS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ikonomovska et al., 2007] Elena Ikonomovska, Suzana Loskovska, and Dejan Gjorgjevik. A survey of stream data mining. In ETAI, pages 19-21, 2007.","title":"A survey of stream data mining","authors":"Elena Ikonomovska, Suzana Loskovska, Dejan Gjorgjevik","year":"2007","venue":"ETAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Kim et al., 2011] Tae-Kyun Kim, Björn Stenger, Josef Kit- tler, and Roberto Cipolla. Incremental linear discriminant analysis using sufficient spanning sets and its applications. IJCV, 91(2), 2011.","title":"Incremental linear discriminant analysis using sufficient spanning sets and its applications","authors":"Kim","year":"2011","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Law et al., 2004] Martin HC Law, Nan Zhang, and Anil K Jain. Nonlinear manifold learning for data stream. In ICDM, 2004.","title":"Nonlinear manifold learning for data stream","authors":"Nan Martin Hc Law, Anil K Zhang, Jain","year":"2004","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Li et al., 2004] Haifeng Li, Tao Jiang, and Keshu Zhang. Efficient and robust feature extraction by maximum mar- gin criterion. In NIPS, pages 97-104, 2004.","title":"Efficient and robust feature extraction by maximum margin criterion","authors":"Haifeng Li, Tao Jiang, Keshu Zhang","year":"2004","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Liu and Motoda, 1998] Huan Liu and Hiroshi Motoda. Fea- ture extraction, construction and selection: A data mining perspective, volume 453. Springer, 1998.","title":"Feature extraction, construction and selection: A data mining perspective","authors":"Huan Liu, Hiroshi Motoda","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Maaten and Hinton, 2008] Laurens van der Maaten and Ge- offrey Hinton. Visualizing data using t-sne. JMLR, 2008.","title":"Laurens van der Maaten and Geoffrey Hinton","authors":"Hinton Maaten","year":"2008","venue":"Visualizing data using t-sne","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[McInnes et al., 2018] Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. 2018.","title":"Umap: Uniform manifold approximation and projection for dimension reduction","authors":"Leland Mcinnes, John Healy, James Melville","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[McLachlan, 2004] Geoffrey J McLachlan. Discriminant analysis and statistical pattern recognition. Wiley, 2004. [Mitliagkas et al., 2013]","title":"Discriminant analysis and statistical pattern recognition","authors":"Geoffrey J Mclachlan","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"Ioannis Mitliagkas, Constantine Caramanis, and Prateek Jain. Memory limited, streaming pca. In NIPS, pages 2886-2894, 2013.","title":"Memory limited, streaming pca","authors":"Ioannis Mitliagkas, Constantine Caramanis, Prateek Jain","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Pang et al., 2005] Shaoning Pang, Seiichi Ozawa, and Nikola Kasabov. Incremental linear discriminant analysis for classification of data streams. TSMC, 35(5), 2005.","title":"Incremental linear discriminant analysis for classification of data streams","authors":"Pang","year":"2005","venue":"TSMC","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Qiu et al., 2009] Chenlu Qiu, Wei Lu, and Namrata Vaswani. Real-time dynamic mr image reconstruction us- ing kalman filtered compressed sensing. In ICASSP, 2009.","title":"Real-time dynamic mr image reconstruction using kalman filtered compressed sensing","authors":"Qiu","year":"2009","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Ross et al., 2008] David A Ross, Jongwoo Lim, Ruei-Sung Lin, and Ming-Hsuan Yang. Incremental learning for ro- bust visual tracking. IJCV, 77(1-3):125-141, 2008.","title":"Incremental learning for robust visual tracking","authors":"Jongwoo David A Ross, Ruei-Sung Lim, Ming-Hsuan Lin, Yang","year":"2008","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Schoeneman et al., 2017] Frank Schoeneman, Suchismit Mahapatra, Varun Chandola, Nils Napp, and Jaroslaw Zola. Error metrics for learning reliable manifolds from streaming data. In ICDM, pages 750-758. SIAM, 2017.","title":"Error metrics for learning reliable manifolds from streaming data","authors":"Frank Schoeneman, Suchismit Mahapatra, Varun Chandola, Nils Napp, Jaroslaw Zola","year":"2017","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Sorzano et al., 2014] Carlos Oscar Sánchez Sorzano, Javier Vargas, and A Pascual Montano. A survey of dimension- ality reduction techniques. 2014.","title":"A survey of dimensionality reduction techniques","authors":"Carlos Oscar, Sánchez Sorzano, Javier Vargas, Montano","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Van Der Maaten et al., 2009] Laurens Van Der Maaten, Eric Postma, and Jaap Van den Herik. Dimensionality re- duction: a comparative. JMLR, 10(66-71):13, 2009.","title":"Dimensionality reduction: a comparative","authors":"Van Der Maaten","year":"2009","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Vempala, 2005] Santosh S Vempala. The random projection method, volume 65. American Mathematical Soc, 2005.","title":"The random projection method","authors":"S Santosh, Vempala","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Vincent et al., 2008] Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extract- ing and composing robust features with denoising autoen- coders. In ICML, pages 1096-1103, 2008.","title":"Extracting and composing robust features with denoising autoencoders","authors":"Pascal Vincent, Hugo Larochelle, Yoshua Bengio, Pierre-Antoine Manzagol","year":"2008","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Weinberger et al., 2009] Kilian Weinberger, Anirban Das- gupta, John Langford, Alex Smola, and Josh Attenberg. Feature hashing for large scale multitask learning. In ICML, pages 1113-1120, 2009.","title":"Feature hashing for large scale multitask learning","authors":"Kilian Weinberger, Anirban Dasgupta, John Langford, Alex Smola, Josh Attenberg","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Weng et al., 2003] Juyang Weng, Yilu Zhang, and Wey- Shiuan Hwang. Candid covariance-free incremental prin- cipal component analysis. TPAMI, 2003.","title":"Candid covariance-free incremental principal component analysis","authors":"Weng","year":"2003","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Wickelmaier, 2003] Florian Wickelmaier. An introduction to mds. Sound Quality Research Unit, 46(5):1-26, 2003.","title":"Florian Wickelmaier. An introduction to mds","authors":"Wickelmaier","year":"2003","venue":"Sound Quality Research Unit","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Yan et al., 2004] Jun Yan, Benyu Zhang, Shuicheng Yan, Qiang Yang, Hua Li, Zheng Chen, Wensi Xi, Weiguo Fan, Wei-Ying Ma, and Qiansheng Cheng. Immc: incremental maximum margin criterion. In SIGKDD, 2004.","title":"Immc: incremental maximum margin criterion","authors":"Jun Yan, Benyu Zhang, Shuicheng Yan, Qiang Yang, Hua Li, Zheng Chen, Wensi Xi, Weiguo Fan, Wei-Ying Ma, Qiansheng Cheng","year":"2004","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Ye et al., 2005] Jieping Ye, Qi Li, Hui Xiong, Haesun Park, Ravi Janardan, and Vipin Kumar. Idr/qr: An incremen- tal dimension reduction algorithm via qr decomposition. TKDE, pages 1208-1222, 2005.","title":"Idr/qr: An incremental dimension reduction algorithm via qr decomposition","authors":"Ye","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Yu et al., 2017] Wenjian Yu, Yu Gu, Jian Li, Shenghua Liu, and Yaohang Li. Single-pass pca of large high- dimensional data. IJCAI, pages 1453-1461, 2017.","title":"Single-pass pca of large highdimensional data","authors":"Wenjian Yu, Yu Gu, Jian Li, Shenghua Liu, Yaohang Li","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhang et al., 2018] Xi Zhang, Hao Huang, Klaus Mueller, and Shinjae Yoo. Streaming classical multidimensional scaling. In NYSDS, pages 1-2. IEEE, 2018.","title":"Streaming classical multidimensional scaling","authors":"Xi Zhang, Hao Huang, Klaus Mueller, Shinjae Yoo","year":"2018","venue":"NYSDS","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Zhou et al., 2012] Guanyu Zhou, Kihyuk Sohn, and Honglak Lee. Online incremental feature learning with denoising autoencoders. In AISTATS, 2012.","title":"Online incremental feature learning with denoising autoencoders","authors":"Zhou","year":"2012","venue":"AISTATS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}