{"type":"ScholarlyArticle","title":"Fast Factorization-free Kernel Learning for Unlabeled Chunk Data Streams","authors":[{"name":"Yi Wang","slug":"yi-wang","given":"Yi","family":"Wang","affiliation":"DUT-RU International School of Information Science and Engineering , Dalian University of Technology , P. R. China; Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province , P. R. China"},{"name":"Nan Xue","slug":"nan-xue","given":"Nan","family":"Xue","affiliation":"DUT-RU International School of Information Science and Engineering , Dalian University of Technology , P. R. China; Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province , P. R. China"},{"name":"Xin Fan","slug":"xin-fan","given":"Xin","family":"Fan","affiliation":"DUT-RU International School of Information Science and Engineering , Dalian University of Technology , P. R. China; Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province , P. R. China"},{"name":"Jiebo Luo","slug":"jiebo-luo","given":"Jiebo","family":"Luo","affiliation":"Department of Computer Science , University of Rochester , USA"},{"name":"Risheng Liu","slug":"risheng-liu","given":"Risheng","family":"Liu","affiliation":"DUT-RU International School of Information Science and Engineering , Dalian University of Technology , P. R. China; Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province , P. R. China"},{"name":"Bin Chen","slug":"bin-chen","given":"Bin","family":"Chen","affiliation":"DUT-RU International School of Information Science and Engineering , Dalian University of Technology , P. R. China; Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province , P. R. China"},{"name":"Haojie Li","slug":"haojie-li","given":"Haojie","family":"Li","affiliation":"DUT-RU International School of Information Science and Engineering , Dalian University of Technology , P. R. China; Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province , P. R. China"},{"name":"Zhongxuan Luo","slug":"zhongxuan-luo","given":"Zhongxuan","family":"Luo","affiliation":"DUT-RU International School of Information Science and Engineering , Dalian University of Technology , P. R. China; Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province , P. R. China"}],"abstract":"Data stream analysis aims at extracting discriminative information for classification from continuously incoming samples. It is extremely challenging to detect novel data while updating the model in an efficient and stable fashion, especially for the chunk data. This paper proposes a fast factorization-free kernel learning method to unify novelty detection and incremental learning for unlabeled chunk data streams in one framework. The proposed method constructs a joint reproducing kernel Hilbert space from known class centers by solving a linear system in kernel space. Naturally, unlabeled data can be detected and classified among multi-classes by a single decision model. And projecting samples into the discriminative feature space turns out to be the product of two small-sized kernel matrices without needing such time-consuming factorization like QR-decomposition or singular value decomposition. Moreover, the insertion of a novel class can be treated as the addition of a new orthogonal basis\r\nto the existing feature space, resulting in fast and stable updating schemes. Both theoretical analysis and experimental validation on real-world datasets demonstrate that the proposed methods learn chunk data streams with significantly lower computational costs and comparable or superior accuracy than the state of the art.","doi":"10.24963/ijcai.2018/393","url":"https://ijcai.org/proceedings/2018/393","pdf":"https://ijcai.org/proceedings/2018/0393.pdf","markdown":"https://ijcai.org/proceedings/2018/393.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/393","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":393,"pages":{"first":2833,"last":2839},"keywords":["Machine Learning: Kernel Methods","Machine Learning: Online Learning","Machine Learning: Classification","Machine Learning: Semi-Supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Aggarwal et al., 2003] Charu C Aggarwal, Jiawei Han, Jianyong Wang, and Philip S Yu. A framework for cluster- ing evolving data streams. In Proceedings of the 29th inter- national conference on Very large data bases, volume 29, pages 81-92. VLDB Endowment, 2003.","title":"A framework for clustering evolving data streams","authors":"Jiawei Charu C Aggarwal, Jianyong Han, Philip S Wang, Yu","year":"2003","venue":"Proceedings of the 29th international conference on Very large data bases","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bodesheim et al., 2013] Paul Bodesheim, Alexander Frey- tag, Erik Rodner, Michael Kemmler, and Joachim Denzler. Kernel null space methods for novelty detection. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR-13), pages 3374-3381, 2013.","title":"Kernel null space methods for novelty detection","authors":"Paul Bodesheim, Alexander Freytag, Erik Rodner, Michael Kemmler, Joachim Denzler","year":"2013","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR-13)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2007] Deng Cai, Xiaofei He, and Jiawei Han. Ef- ficient kernel discriminant analysis via spectral regression. In Proceedings of Seventh IEEE International Conference on Data Mining (ICDM -07), pages 427-432. IEEE, 2007.","title":"Efficient kernel discriminant analysis via spectral regression","authors":"Deng Cai, Xiaofei He, Jiawei Han","year":"2007","venue":"Proceedings of Seventh IEEE International Conference on Data Mining (ICDM -07)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2000] Li-Fen Chen, Hong-Yuan Mark Liao, Ming-Tat Ko, Ja-Chen Lin, and Gwo-Jong Yu. A new lda- based face recognition system which can solve the small sample size problem. Pattern recognition, 33(10):1713- 1726, 2000.","title":"A new ldabased face recognition system which can solve the small sample size problem","authors":"Li-Fen Chen, Hong-Yuan Mark Liao, Ming-Tat Ko, Ja-Chen Lin, Gwo-Jong Yu","year":"2000","venue":"Pattern recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chu et al., 2015] Delin Chu, Li-Zhi Liao, Michael Kwok- Po Ng, and Xiaoyan Wang. Incremental linear discrimi- nant analysis: a fast algorithm and comparisons. IEEE transactions on neural networks and learning systems, 26(11):2716-2735, 2015.","title":"Incremental linear discriminant analysis: a fast algorithm and comparisons","authors":"Delin Chu, Li-Zhi Liao, Michael Kwok-Po, Xiaoyan Ng, Wang","year":"2015","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Faria et al., 2016] Elaine R Faria, Isabel JCR Gonçalves, An- dré CPLF de Carvalho, and João Gama. Novelty detection in data streams. Artificial Intelligence Review, 45(2):235- 269, 2016.","title":"Novelty detection in data streams","authors":"Isabel Jcr Elaine R Faria, André Cplf De Gonçalves, João Carvalho, Gama","year":"2016","venue":"Artificial Intelligence Review","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Griffin et al., 2007] Gregory Griffin, Alex Holub, and Pietro Perona. Caltech-256 object category dataset. 2007.","title":"Caltech-256 object category dataset","authors":"Gregory Griffin, Alex Holub, Pietro Perona","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2002] Rui Huang, Qingshan Liu, Hanqing Lu, and Songde Ma. Solving the small sample size problem of lda. In Proceedings of the 16th International Conference on Pattern Recognition, volume 3, pages 29-32. IEEE, 2002.","title":"Solving the small sample size problem of lda","authors":"Rui Huang, Qingshan Liu, Hanqing Lu, Songde Ma","year":"2002","venue":"Proceedings of the 16th International Conference on Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kemmler et al., 2010] Michael Kemmler, Erik Rodner, and Joachim Denzler. One-class classification with gaussian processes. In Asian Conference on Computer Vision (ACCV- 10), pages 489-500. Springer, 2010.","title":"One-class classification with gaussian processes","authors":"Kemmler","year":"2010","venue":"Asian Conference on Computer Vision (ACCV-10)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kim et al., 2011] Tae-Kyun Kim, Björn Stenger, Josef Kit- tler, and Roberto Cipolla. Incremental linear discriminant analysis using sufficient spanning sets and its applications. International Journal of Computer Vision, 91(2):216-232, 2011.","title":"Incremental linear discriminant analysis using sufficient spanning sets and its applications","authors":"Kim","year":"2011","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural in- formation processing systems, pages 1097-1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lampert et al., 2014] Christoph H Lampert, Hannes Nick- isch, and Stefan Harmeling. Attribute-based classification for zero-shot visual object categorization. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 36(3):453-465, 2014.","title":"Attribute-based classification for zero-shot visual object categorization","authors":"Hannes Christoph H Lampert, Stefan Nickisch, Harmeling","year":"2014","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2017] Juncheng Liu, Zhouhui Lian, Yi Wang, and Jianguo Xiao. Incremental kernel null space discriminant analysis for novelty detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR-17), pages 792-800, 2017.","title":"Incremental kernel null space discriminant analysis for novelty detection","authors":"Liu","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR-17)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lu et al., 2017] Yang Lu, Yiu-ming Cheyng, and Yuan Yan Tang. Dynamic weighted majority for incremental learning of imbalanced data streams with concept drift. In Proceed- ings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17), pages 2393-2399, 2017.","title":"Dynamic weighted majority for incremental learning of imbalanced data streams with concept drift","authors":"Yang Lu, Yiu-Ming Cheyng, Yuan Yan Tang","year":"2017","venue":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17)","doi":null,"url":null,"internal_url":"/proceedings/2017/333"},{"position":15,"raw":"[Min et al., 2016] Hwang-Ki Min, Yuxi Hou, Sangwoo Park, and Iickho Song. A computationally efficient scheme for feature extraction with kernel discriminant analysis. Pattern Recognition, 50:45-55, 2016.","title":"A computationally efficient scheme for feature extraction with kernel discriminant analysis","authors":"Yuxi Hwang-Ki Min, Sangwoo Hou, Iickho Park, Song","year":"2016","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tax and Duin, 2004] David MJ Tax and Robert PW Duin. Support vector data description. Machine learning, 54(1):45-66, 2004.","title":"Support vector data description","authors":"M J David, Tax, Robert Pw Duin","year":"2004","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2016] Shuo Wang, Leandro L Minku, and Xin Yao. Dealing with multiple classes in online class imbal- ance learning. In Proceedings of the Twenty-Fifth Interna- tional Joint Conference on Artificial Intelligence (IJCAI- 16), pages 2118-2124, 2016.","title":"Dealing with multiple classes in online class imbalance learning","authors":"Shuo Wang, Leandro L Minku, Xin Yao","year":"2016","venue":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16)","doi":null,"url":null,"internal_url":"/proceedings/2016/302"},{"position":18,"raw":"[Xiao et al., 2017] Han Xiao, Kashif Rasul, and Roland Voll- graf. Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.","title":"Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms","authors":"Han Xiao, Kashif Rasul, Roland Vollgraf","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xiong et al., 2005] Tao Xiong, Jieping Ye, Qi Li, Ravi Ja- nardan, and Vladimir Cherkassky. Efficient kernel discrim- inant analysis via qr decomposition. In Advances in neural information processing systems, pages 1529-1536, 2005.","title":"Efficient kernel discriminant analysis via qr decomposition","authors":"Tao Xiong, Jieping Ye, Qi Li, Ravi Janardan, Vladimir Cherkassky","year":"2005","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}