{"type":"ScholarlyArticle","title":"Approximate Large-scale Multiple Kernel k-means Using Deep Neural Network","authors":[{"name":"Yueqing Wang","slug":"yueqing-wang","given":"Yueqing","family":"Wang","affiliation":"National Laboratory for Parallel and Distributed Processing , NUDT , Changsha , China , 410073"},{"name":"Xinwang Liu","slug":"xinwang-liu","given":"Xinwang","family":"Liu","affiliation":"National Laboratory for Parallel and Distributed Processing , NUDT , Changsha , China , 410073"},{"name":"Yong Dou","slug":"yong-dou","given":"Yong","family":"Dou","affiliation":"National Laboratory for Parallel and Distributed Processing , NUDT , Changsha , China , 410073"},{"name":"Rongchun Li","slug":"rongchun-li","given":"Rongchun","family":"Li","affiliation":"National Laboratory for Parallel and Distributed Processing , NUDT , Changsha , China , 410073"}],"abstract":"Multiple kernel clustering (MKC) algorithms have been extensively studied and applied to various applications. Although they demonstrate great success in both the theoretical aspects and applications, existing MKC algorithms cannot be applied to large-scale clustering tasks due to: i) the heavy computational cost to calculate the base kernels; and ii) insufficient memory to load the kernel matrices. In this paper, we propose an approximate algorithm to overcome these issues, and to make it be applicable to large-scale applications. Specifically, our algorithm trains a deep neural network to regress the indicating matrix generated by MKC algorithms on a small subset, and then obtains the approximate indicating matrix of the whole data set using the trained network, and finally performs the $k$-means on the output of our network. By mapping features into indicating matrix directly, our algorithm avoids computing the full kernel matrices, which dramatically decreases the memory requirement. Extensive experiments show that our algorithm consumes less time than most comparatively similar algorithms, while it achieves comparable performance with MKC algorithms.","doi":"10.24963/ijcai.2017/419","url":"https://ijcai.org/proceedings/2017/419","pdf":"https://ijcai.org/proceedings/2017/0419.pdf","markdown":"https://ijcai.org/proceedings/2017/419.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/419","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":419,"pages":{"first":3006,"last":3012},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Kernel Methods","Machine Learning: Multi-instance/Multi-label/Multi-view learning"],"videos":[],"references":[{"position":1,"raw":"[Cai et al., 2013] Xiao Cai, Feiping Nie, and Heng Huang. Multi-view k-means clustering on big data. In IJCAI, pages 2598-2604, 2013.","title":"Multi-view k-means clustering on big data","authors":"Xiao Cai, Feiping Nie, Heng Huang","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/383"},{"position":2,"raw":"[Cao et al., 2015] Xiaochun Cao, Changqing Zhang, Huazhu Fu, and Si Liu. Diversity-induced multi-view subspace clustering. In CVPR, pages 586-594, 2015.","title":"Diversity-induced multi-view subspace clustering","authors":"Xiaochun Cao, Changqing Zhang, Huazhu Fu, Si Liu","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and Cai, 2011] Xinlei Chen and Deng Cai. Large s- cale spectral clustering with landmark-based representa- tion. In AAAI, pages 313-318, 2011.","title":"Large scale spectral clustering with landmark-based representation","authors":"Chen, Cai, Xinlei Chen, Deng Cai","year":"2011","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cortes et al., 2012] Corinna Cortes, Mehryar Mohri, and Afshin Rostamizadeh. Algorithms for learning kernel- s based on centered alignment. The Journal of Machine Learning Research, 13(1):795-828, 2012.","title":"Algorithms for learning kernels based on centered alignment","authors":"Corinna Cortes, Mehryar Mohri, Afshin Rostamizadeh","year":"2012","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Du et al., 2015] Liang Du, Peng Zhou, Lei Shi, Hanmo Wang, Mingyu Fan, Wenjian Wang, and Yi Dong Shen. Robust multiple kernel k-means using ℓ 2,1 -norm. In In- ternational Conference on Artificial Intelligence, pages 3476-3482, 2015.","title":"Robust multiple kernel k-means using ℓ 2,1 -norm","authors":"Liang Du, Peng Zhou, Lei Shi, Hanmo Wang, Mingyu Fan, Wenjian Wang, Yi Dong Shen","year":"2015","venue":"ternational Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gönen and Margolin, 2014] Mehmet Gönen and Adam A Margolin. Localized data fusion for kernel k-means clus- tering with application to cancer biology. NIPS, 2:1305- 1313, 2014.","title":"Localized data fusion for kernel k-means clustering with application to cancer biology","authors":"Margolin ; ] Mehmet Gönen, Adam A Gönen, Margolin","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gong et al., 2015] Yunchao Gong, Marcin Pawlowski, Fei Yang, Louis Brandy, Lubomir Boundev, and Rob Fergus. Web scale photo hash clustering on a single machine. In CVPR, pages 19-27, 2015.","title":"Web scale photo hash clustering on a single machine","authors":"Gong","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2012] Hsin Chien Huang, Yung Yu Chuang, and Chu Song Chen. Multiple kernel fuzzy cluster- ing. IEEE Transactions on Fuzzy Systems, 20(1):120-134, 2012.","title":"Multiple kernel fuzzy clustering","authors":"Yung Hsin Chien Huang, Chu Yu Chuang, Chen Song","year":"2012","venue":"IEEE Transactions on Fuzzy Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jinglin et al., 2016] Xu Jinglin, Han Junwei, and Nie Feip- ing. Discriminatively embedded k-means for multi-view clustering. In CVPR, pages 5356-5364, 2016.","title":"Discriminatively embedded k-means for multi-view clustering","authors":"Han Xu Jinglin, Nie Junwei, Feiping","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in Neural Infor- mation Processing Systems, 25(2), 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kumar et al., 2011] Abhishek Kumar, Piyush Rai, and Hal Daume. Co-regularized multi-view spectral clustering. In NIPS, pages 1413-1421, 2011.","title":"Co-regularized multi-view spectral clustering","authors":"Abhishek Kumar, Piyush Rai, Hal Daume","year":"2011","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2015] Yeqing Li, Feiping Nie, Heng Huang, and Junzhou Huang. Large-scale multi-view spectral cluster- ing via bipartite graph. pages 2750-2756, 2015.","title":"Large-scale multi-view spectral clustering via bipartite graph","authors":"Li","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2016] Xinwang Liu, Yong Dou, Jianping Yin, Lei Wang, and En Zhu. Multiple kernel k-means clustering with matrix-induced regularization. In AAAI, pages 1888- 1894, 2016.","title":"Multiple kernel k-means clustering with matrix-induced regularization","authors":"Xinwang Liu, Yong Dou, Jianping Yin, Lei Wang, En Zhu","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lu et al., 2014] Yanting Lu, Liantao Wang, Jianfeng Lu, Jingyu Yang, and Chunhua Shen. Multiple kernel clus- tering based on centered kernel alignment. Pattern Recog- nition, 47(11):3656-3664, 2014.","title":"Multiple kernel clustering based on centered kernel alignment","authors":"Yanting Lu, Liantao Wang, Jianfeng Lu, Jingyu Yang, Chunhua Shen","year":"2014","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nie et al., 2016] Feiping Nie, Jing Li, and Xuelong Li. Parameter-free auto-weighted multiple graph learning: A framework for multiview clustering and semi-supervised classification. In IJCAI, pages 1881-1887, 2016.","title":"Parameter-free auto-weighted multiple graph learning: A framework for multiview clustering and semi-supervised classification","authors":"Nie","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/269"},{"position":16,"raw":"[Nie et al., 2017] Feiping Nie, Guohao Cai, and Xuelong Li. Multi-view clustering and semi-supervised classification with adaptive neighbours. In AAAI, 2017.","title":"Multi-view clustering and semi-supervised classification with adaptive neighbours","authors":"Nie","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shen et al., 2017] Xiaobo Shen, Weiwei Liu, Ivor Tsang, Fuming Shen, and Quan-sen Sun. Compressed k-means for large-scale clustering. In AAAI, 2017.","title":"Compressed k-means for large-scale clustering","authors":"Xiaobo Shen, Weiwei Liu, Ivor Tsang, Fuming Shen, Quan-Sen Sun","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Simonyan and Zisserman, 2014] K. Simonyan and A. Zis- serman. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"K Simonyan, A Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2016] De Wang, Feiping Nie, and Heng Huang. Fast robust non-negative matrix factorization for large- scale human action data clustering. In IJCAI, pages 2104- 2110, 2016.","title":"Fast robust non-negative matrix factorization for largescale human action data clustering","authors":"De Wang, Feiping Nie, Heng Huang","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Weiran et al., 2015] Wang Weiran, Arora Raman, Livescu Karen, and Bilmes Jeff. On deep multi-view representation learning. In ICML, pages 1083-1092, 2015.","title":"On deep multi-view representation learning","authors":"Wang Weiran, Arora Raman, Livescu Karen, Bilmes Jeff","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xia et al., 2014] Rongkai Xia, Yan Pan, Lei Du, and Jian Yin. Robust multi-view spectral clustering via low-rank and sparse decomposition. In AAAI, pages 2149-2155, 2014.","title":"Robust multi-view spectral clustering via low-rank and sparse decomposition","authors":"Xia","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yu et al., 2012] Shi Yu, Lon Charles Tranchevent, Bart De Moor, and Yves Moreau. Optimized data fusion for kernel k-means clustering. IEEE Transactions on Pattern Analy- sis Machine Intelligence, 34(5):1031-1039, 2012.","title":"Optimized data fusion for kernel k-means clustering","authors":"Shi Yu, Lon Charles Tranchevent, Bart De Moor, Yves Moreau","year":"2012","venue":"IEEE Transactions on Pattern Analysis Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang and Lu, 2016] Ruqi Zhang and Zhiwu Lu. Large s- cale sparse clustering. In IJCAI, pages 2336-2342, 2016.","title":"Large scale sparse clustering","authors":"Lu ; Zhang, Ruqi Zhang, Zhiwu Lu","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/333"},{"position":24,"raw":"[Zhang et al., 2015] Changqing Zhang, Huazhu Fu, Si Liu, Guangcan Liu, and Xiaochun Cao. Low-rank tensor con- strained multiview subspace clustering. In ICCV, pages 1582-1590, 2015.","title":"Low-rank tensor constrained multiview subspace clustering","authors":"Changqing Zhang, Huazhu Fu, Si Liu, Guangcan Liu, Xiaochun Cao","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhao et al., 2009] Bin Zhao, James T. Kwok, and Chang- shui Zhang. Multiple kernel clustering. In ICDM, pages 638-649, 2009.","title":"Multiple kernel clustering","authors":"Bin Zhao, James T Kwok, Changshui Zhang","year":"2009","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou and Burges, 2007] Dengyong Zhou and Christopher J. C. Burges. Spectral clustering and transductive learning with multiple views. In ICML, pages 1159-1166, 2007.","title":"Spectral clustering and transductive learning with multiple views","authors":"Burges Zhou, Dengyong Zhou, Christopher J C Burges","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2015] Peng Zhou, Liang Du, Lei Shi, Hanmo Wang, and Yi-Dong Shen. Recovery of corrupted multiple kernels for clustering. In Proceedings of the 24th Interna- tional Conference on Artificial Intelligence, pages 4105- 4111, 2015.","title":"Recovery of corrupted multiple kernels for clustering","authors":"Peng Zhou, Liang Du, Lei Shi, Hanmo Wang, Yi-Dong Shen","year":"2015","venue":"Proceedings of the 24th International Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2015/576"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}