{"type":"ScholarlyArticle","title":"Online Learning from Capricious Data Streams: A Generative Approach","authors":[{"name":"Yi He","slug":"yi-he","given":"Yi","family":"He","affiliation":"School of Computing and Informatics , University of Louisiana at Lafayette , USA"},{"name":"Baijun Wu","slug":"baijun-wu","given":"Baijun","family":"Wu","affiliation":"School of Computing and Informatics , University of Louisiana at Lafayette , USA"},{"name":"Di Wu","slug":"di-wu","given":"Di","family":"Wu","affiliation":"Chongqing Institute of Green and Intelligent Technology , Chinese Academy of Sciences , China"},{"name":"Ege Beyazit","slug":"ege-beyazit","given":"Ege","family":"Beyazit","affiliation":"School of Computing and Informatics , University of Louisiana at Lafayette , USA"},{"name":"Sheng Chen","slug":"sheng-chen","given":"Sheng","family":"Chen","affiliation":"School of Computing and Informatics , University of Louisiana at Lafayette , USA"},{"name":"Xindong Wu","slug":"xindong-wu","given":"Xindong","family":"Wu","affiliation":"School of Computing and Informatics , University of Louisiana at Lafayette , USA"}],"abstract":"Learning with streaming data has received extensive attention during the past few years. Existing approaches assume the feature space is fixed or changes by following explicit regularities, limiting their applicability in dynamic environments where the data streams are described by an arbitrarily varying feature space. To handle such capricious data streams, we in this paper develop a novel algorithm, named OCDS (Online learning from Capricious Data Streams), which does not make any assumption on feature space dynamics. OCDS trains a learner on a universal feature space that establishes relationships between old and new features, so that the patterns learned in the old feature space can be used in the new feature space. Specifically, the universal feature space is constructed by leveraging the relatednesses among features. We propose a generative graphical model to model the construction process, and show that learning from the universal feature space can effectively improve performance with theoretical analysis. The experimental results demonstrate that OCDS achieves conspicuous performance on synthetic and real datasets.","doi":"10.24963/ijcai.2019/346","url":"https://ijcai.org/proceedings/2019/346","pdf":"https://ijcai.org/proceedings/2019/0346.pdf","markdown":"https://ijcai.org/proceedings/2019/346.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/346","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":346,"pages":{"first":2491,"last":2497},"keywords":["Machine Learning: Online Learning","Machine Learning: Time-series;Data Streams","Machine Learning: Learning Generative Models"],"videos":[],"references":[{"position":1,"raw":"[Baig and Gholamhosseini, 2013] Mirza Mansoor Baig and Hamid Gholamhosseini. Smart health monitoring systems: an overview of design and modeling. Journal of medical systems, 37(2):9898, 2013.","title":"Smart health monitoring systems: an overview of design and modeling","authors":"Mansoor Mirza, Hamid Baig, Gholamhosseini","year":"2013","venue":"Journal of medical systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Beyazit et al., 2018] Ege Beyazit, Matin Hosseini, Anthony Maida, and Xindong Wu. Learning simplified decision boundaries from trapezoidal data streams. In ICANN, pages 508-517. Springer, 2018.","title":"Learning simplified decision boundaries from trapezoidal data streams","authors":"Ege Beyazit, Matin Hosseini, Anthony Maida, Xindong Wu","year":"2018","venue":"ICANN","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Beyazit et al., 2019] Ege Beyazit, Jeevithan Alagurajah, and Xindong Wu. Online learning from data streams with varying feature spaces. In AAAI, 2019.","title":"Online learning from data streams with varying feature spaces","authors":"Ege Beyazit, Jeevithan Alagurajah, Xindong Wu","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cesa-Bianchi and Lugosi, 2006] Nicolo Cesa-Bianchi and Gábor Lugosi. Prediction, learning, and games. Cam- bridge university press, 2006.","title":"Nicolo Cesa-Bianchi and Gábor Lugosi","authors":"Cesa-Bianchi, Lugosi","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2005] Jinxiu Chen, Donghong Ji, Chew Lim Tan, and Zhengyu Niu. Unsupervised feature selection for relation extraction. In CVPR, 2005.","title":"Unsupervised feature selection for relation extraction","authors":"Jinxiu Chen, Donghong Ji, Chew Lim Tan, Zhengyu Niu","year":"2005","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Crammer et al., 2006] Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, and Yoram Singer. Online passive-aggressive algorithms. Journal of Machine Learn- ing Research, 7(Mar):551-585, 2006.","title":"Online passive-aggressive algorithms","authors":"Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, Yoram Singer","year":"2006","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dua and Karra Taniskidou, 2017] Dheeru Dua and Efi Karra Taniskidou. UCI machine learning repository, 2017.","title":"Dheeru Dua and Efi Karra Taniskidou. UCI machine learning repository","authors":"Karra Dua, Taniskidou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Duda et al., 2012] Richard O Duda, Peter E Hart, and David G Stork. Pattern classification. John Wiley & Sons, 2012.","title":"Pattern classification","authors":"Peter E Richard O Duda, David G Hart, Stork","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gama and Rodrigues, 2009] João Gama and Pedro Pereira Rodrigues. An overview on mining data streams. In Foun- dations of Computational, IntelligenceVolume 6, pages 29-45. Springer, 2009.","title":"João Gama and Pedro Pereira Rodrigues. An overview on mining data streams","authors":"Rodrigues Gama","year":"2009","venue":"Foundations of Computational, IntelligenceVolume","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gerven et al., 2009] Marcel V Gerven, Botond Cseke, Robert Oostenveld, and Tom Heskes. Bayesian source lo- calization with the multivariate laplace prior. In NeurIPS, pages 1901-1909, 2009.","title":"Bayesian source localization with the multivariate laplace prior","authors":"Botond Marcel V Gerven, Robert Cseke, Tom Oostenveld, Heskes","year":"2009","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2019] Yi He, Baijun Wu, Di Wu, Ege Beyazit, Sheng Chen, and Xindong Wu. Supplementary Material of \"Online Learning from Capricious Data Streams: A Generative Approach\". 10.6084/m9.figshare.8316641.v1, 2019.","title":"Supplementary Material of \"Online Learning from Capricious Data Streams: A Generative Approach","authors":"Yi He, Baijun Wu, Di Wu, Ege Beyazit, Sheng Chen, Xindong Wu","year":"2019","venue":null,"doi":"10.6084/m9.figshare.8316641.v1","url":null,"internal_url":null},{"position":12,"raw":"[Hou et al., 2017] Bo-Jian Hou, Lijun Zhang, and Zhi-Hua Zhou. Learning with feature evolvable streams. In NeurIPS, pages 1417-1427, 2017.","title":"Learning with feature evolvable streams","authors":"Bo-Jian Hou, Lijun Zhang, Zhi-Hua Zhou","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2018] Sheng-Jun Huang, Miao Xu, Ming-Kun Xie, Masashi Sugiyama, Gang Niu, and Songcan Chen. Active feature acquisition with supervised matrix comple- tion. SIGKDD, 2018.","title":"Active feature acquisition with supervised matrix completion","authors":"Sheng-Jun Huang, Miao Xu, Ming-Kun Xie, Masashi Sugiyama, Gang Niu, Songcan Chen","year":"2018","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2017] Jundong Li, Jiliang Tang, and Huan Liu. Reconstruction-based unsupervised feature selection: an embedded approach. In IJCAI, 2017.","title":"Reconstruction-based unsupervised feature selection: an embedded approach","authors":"Li","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/300"},{"position":15,"raw":"[Liu et al., 2014] Lingqiao Liu, Chunhua Shen, Lei Wang, Anton Van Den Hengel, and Chao Wang. Encoding high dimensional local features by sparse coding based fisher vectors. In NeurIPS, pages 1143-1151, 2014.","title":"Encoding high dimensional local features by sparse coding based fisher vectors","authors":"Lingqiao Liu, Chunhua Shen, Lei Wang, Anton Van Den, Chao Hengel, Wang","year":"2014","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Maas et al., 2011] Andrew L. Maas, Raymond E. Daly, Pe- ter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In ACL, pages 142-150, Portland, Oregon, USA, June 2011.","title":"Learning word vectors for sentiment analysis","authors":"Andrew L Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, Christopher Potts","year":"2011","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Masud et al., 2013] Mohammad M Masud, Qing Chen, Lat- ifur Khan, Charu C Aggarwal, Jing Gao, Jiawei Han, Ashok Srivastava, and Nikunj C Oza. Classification and adaptive novel class detection of feature-evolving data streams. IEEE Transactions on Knowledge and Data En- gineering, 25(7):1484-1497, 2013.","title":"Classification and adaptive novel class detection of feature-evolving data streams","authors":"Qing Mohammad M Masud, Latifur Chen, Khan, C Charu, Jing Aggarwal, Jiawei Gao, Ashok Han, Nikunj C Srivastava, Oza","year":"2013","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nguyen et al., 2017] Tu Dinh Nguyen, Trung Le, Hung Bui, and Dinh Q Phung. Large-scale online kernel learning with random feature reparameterization. In IJCAI, pages 2543- 2549, 2017.","title":"Large-scale online kernel learning with random feature reparameterization","authors":"Trung Tu Dinh Nguyen, Hung Le, Dinh Q Bui, Phung","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/354"},{"position":19,"raw":"[Pan et al., 2010] Sinno Jialin Pan, Qiang Yang, et al. A sur- vey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345-1359, 2010.","title":"A survey on transfer learning","authors":"Qiang Sinno Jialin Pan, Yang","year":"2010","venue":"IEEE Transactions on knowledge and data engineering","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Park and Lee, 2005] Hyun J Park and Te W Lee. Modeling nonlinear dependencies in natural images using mixture of laplacian distribution. In NeurIPS, pages 1041-1048, 2005.","title":"Modeling nonlinear dependencies in natural images using mixture of laplacian distribution","authors":"J Hyun, Te W Park, Lee","year":"2005","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sáez et al., 2014] José A Sáez, Mikel Galar, Julián Luengo, and Francisco Herrera. Analyzing the presence of noise in multi-class problems: alleviating its influence with the one-vs-one decomposition. Knowledge and information systems, 38(1):179-206, 2014.","title":"Analyzing the presence of noise in multi-class problems: alleviating its influence with the one-vs-one decomposition","authors":"Sáez","year":"2014","venue":"Knowledge and information systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sun, 2013] Shiliang Sun. A survey of multi-view ma- chine learning. Neural Computing and Applications, 23(7- 8):2031-2038, 2013.","title":"A survey of multi-view machine learning","authors":"Shiliang Sun","year":"2013","venue":"Neural Computing and Applications","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tseng and Yun, 2009] Paul Tseng and Sangwoon Yun. A coordinate gradient descent method for nonsmooth sepa- rable minimization. Mathematical Programming, 117(1- 2):387-423, 2009.","title":"A coordinate gradient descent method for nonsmooth separable minimization","authors":"Yun ; ] Paul Tseng, Sangwoon Tseng, Yun","year":"2009","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xu, 2011] Jianhua Xu. An extended one-versus-rest support vector machine for multi-label classification. Neurocom- puting, 74(17):3114-3124, 2011.","title":"An extended one-versus-rest support vector machine for multi-label classification","authors":"Jianhua Xu","year":"2011","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2013] Wei Zhang, Wei Feng, and Jianyong Wang. Integrating semantic relatedness and words' in- trinsic features for keyword extraction. In IJCAI, pages 2225-2231, 2013.","title":"Integrating semantic relatedness and words' intrinsic features for keyword extraction","authors":"Wei Zhang, Wei Feng, Jianyong Wang","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2016a] Qin Zhang, Peng Zhang, Guodong Long, Wei Ding, Chengqi Zhang, and Xindong Wu. On- line learning from trapezoidal data streams. IEEE Transac- tions on Knowledge and Data Engineering, 28(10):2709- 2723, 2016.","title":"Online learning from trapezoidal data streams","authors":"Zhang","year":"2016","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2016b] Xianchao Zhang, Xiaotong Zhang, and Han Liu. Self-adapted multi-task clustering. In IJCAI, pages 2357-2363, 2016.","title":"Self-adapted multi-task clustering","authors":"Zhang","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2012] Guanyu Zhou, Kihyuk Sohn, and Honglak Lee. Online incremental feature learning with denoising autoencoders. In AISTATS, pages 1453-1461, 2012.","title":"Online incremental feature learning with denoising autoencoders","authors":"Zhou","year":"2012","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zinkevich, 2003] Martin Zinkevich. Online convex pro- gramming and generalized infinitesimal gradient ascent. In ICML, pages 928-936, 2003.","title":"Online convex programming and generalized infinitesimal gradient ascent","authors":"Martin Zinkevich","year":"2003","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}