{"type":"ScholarlyArticle","title":"Large-scale Subspace Clustering by Fast Regression Coding","authors":[{"name":"Jun Li","slug":"jun-li","given":"Jun","family":"Li","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA , 02115 , USA."},{"name":"Handong Zhao","slug":"handong-zhao","given":"Handong","family":"Zhao","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA , 02115 , USA."},{"name":"Zhiqiang Tao","slug":"zhiqiang-tao","given":"Zhiqiang","family":"Tao","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA , 02115 , USA."},{"name":"Yun Fu","slug":"yun-fu","given":"Yun","family":"Fu","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA , 02115 , USA.; College of Computer and Information Science , Northeastern University , Boston , MA , 02115 , USA."}],"abstract":"Large-Scale Subspace Clustering (LSSC) is an interesting and important problem in big data era. However, most existing methods (i.e., sparse or low-rank subspace clustering) cannot be directly used for solving LSSC because they suffer from the high time complexity-quadratic or cubic in n (the number of data points). To overcome this limitation, we propose a Fast Regression Coding (FRC) to optimize regression codes, and simultaneously train a non-linear function to approximate the codes. By using FRC, we develop an efficient Regression Coding Clustering (RCC) framework to solve the LSSC problem. It consists of sampling, FRC and clustering. RCC randomly samples a small number of data points, quickly calculates the codes of all data points by using the non-linear function learned from FRC, and employs a large-scale spectral clustering method to cluster the codes. Besides, we provide a theorem guarantee that the non-linear function has a first-order approximation ability and a group effect. The theorem manifests that the codes are easily used to construct a dividable similarity graph. Compared with the state-of-the-art LSSC methods, our model achieves better clustering results in large-scale datasets.","doi":"10.24963/ijcai.2017/297","url":"https://ijcai.org/proceedings/2017/297","pdf":"https://ijcai.org/proceedings/2017/0297.pdf","markdown":"https://ijcai.org/proceedings/2017/297.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/297","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":297,"pages":{"first":2138,"last":2144},"keywords":["Robotics and Vision: Vision and Perception","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Bengio et al., 2013] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell., 35(8):1798-1828, Aug. 2013.","title":"Representation learning: A review and new perspectives","authors":"Yoshua Bengio, Aaron Courville, Pascal Vincent","year":"2013","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai and Chen, 2015] Deng Cai and Xinlei Chen. Large scale spec- tral clustering via landmark-based sparse representation. IEEE Trans. on Cybernetics, 45(8):1669-1680, 2015.","title":"Large scale spectral clustering via landmark-based sparse representation","authors":"Chen ; ] Deng Cai, Xinlei Cai, Chen","year":"2015","venue":"IEEE Trans. on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai, 2011] Deng Cai. Litekmeans: the fastest mat- lab implementation of kmeans. In Available at: http://www.zjucadcg.cn/dengcai/Data/Clustering.html, 2011.","title":"Litekmeans: the fastest matlab implementation of kmeans","authors":"Deng Cai","year":"2011","venue":null,"doi":null,"url":"http://www.zjucadcg.cn/dengcai/Data/Clustering.html","internal_url":null},{"position":4,"raw":"[Chen and Cai, 2011] Xinlei Chen and Deng Cai. Large scale spec- tral clustering with landmark-based representation. In Proc. of the AAAI Conf. on Artif. Intell., pages 313-318, 2011.","title":"Large scale spectral clustering with landmark-based representation","authors":"Chen, Cai ; ] Xinlei Chen, Deng Cai","year":"2011","venue":"Proc. of the AAAI Conf. on Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2011] Wen-Yen Chen, Yangqiu Song, Hongjie Bai, Chih-Jen Lin, and Edward Y. Chang. Parallel spectral clustering in distributed systems. IEEE Trans. Pattern Anal. Mach. Intell., 33(3):568-586, 2011.","title":"Parallel spectral clustering in distributed systems","authors":"Wen-Yen Chen, Yangqiu Song, Hongjie Bai, Chih-Jen Lin, Edward Y Chang","year":"2011","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Elhamifar and Vidal, 2013] Ehsan Elhamifar and Rene Vidal. S- parse subspace clustering: Algorithm, theory, and applications. IEEE Trans. Pattern Anal. Mach. Intell., 35:2765-2781, 2013.","title":"Sparse subspace clustering: Algorithm, theory, and applications","authors":"Ehsan Elhamifar, Rene Vidal","year":"2013","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fowlkes et al., 2004] Charless Fowlkes, Serge Belongie, Fan Chung, and Jitendra Malik. Spectral grouping using the nystrom method. IEEE Trans. Pattern Anal. Mach. Intell., 26(2):214-225, 2004.","title":"Spectral grouping using the nystrom method","authors":"Charless Fowlkes, Serge Belongie, Fan Chung, Jitendra Malik","year":"2004","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gregor and LeCun, 2010] Karol Gregor and Yann LeCun. Learning fast approximations of sparse coding. In Proc. of Int. Conf. Mach. Learn., pages 399-406, 2010.","title":"Learning fast approximations of sparse coding","authors":"Karol Gregor, Yann Lecun","year":"2010","venue":"Proc. of Int. Conf. Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Haykin, 2009] Simon Haykin. In Neural Networks and Learning Machines. Pearson Education Inc., 2009.","title":"Neural Networks and Learning Machines","authors":"Simon Haykin","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hoerl and Kennard, 2000] Arthur E. Hoerl and Robert W. Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 42(1):80-86, 2000.","title":"Ridge regression: Biased estimation for nonorthogonal problems","authors":"Arthur E Hoerl, Robert W Kennard","year":"2000","venue":"Technometrics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kavukcuoglu et al., 2008] K. Kavukcuoglu, M. Ranzato, and Y. Le- Cun. Fast inference in sparse coding algorithms with applications to object recognition. In CBLL-TR-2008-12-01, 2008.","title":"Fast inference in sparse coding algorithms with applications to object recognition","authors":"K Kavukcuoglu, M Ranzato, Y Le-Cun","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lee et al., 2015] Minsik Lee, Jieun Lee, Hyeogjin Lee, and Nojun Kwak. Membership representation for detecting block-diagonal structure in low-rank or sparse subspace clustering. In Proc. of IEEE Conf. Comput. Vis. Pattern Recognit., pages 1648-1656, 2015.","title":"Membership representation for detecting block-diagonal structure in low-rank or sparse subspace clustering","authors":"Lee","year":"2015","venue":"Proc. of IEEE Conf. Comput. Vis. Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li and Vidal, 2015] Chun-Guang Li and Rene Vidal. Structured sparse subspace clustering: A unified optimization framework. In Proc. of IEEE Conf. Comput. Vis. Pattern Recognit., pages 277-286, 2015.","title":"Structured sparse subspace clustering: A unified optimization framework","authors":"Chun-Guang Li, Rene Vidal","year":"2015","venue":"Proc. of IEEE Conf. Comput. Vis. Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2015] Jun Li, Heyou Chang, and Jian Yang. Sparse deep stacking network for image classification. In Proc. of the AAAI Conf. on Artif. Intell., pages 3804-3810, 2015.","title":"Sparse deep stacking network for image classification","authors":"Li","year":"2015","venue":"Proc. of the AAAI Conf. on Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2016] Jun Li, Yu Kong, Handong Zhao, Jian Yang, and Yun Fu. Learning fast low-rank projection for image classification. IEEE Trans. Image Process., 25(10):4803-4814, 2016.","title":"Learning fast low-rank projection for image classification","authors":"Jun Li, Yu Kong, Handong Zhao, Jian Yang, Yun Fu","year":"2016","venue":"IEEE Trans. Image Process","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2017a] Jun Li, Yu Kong, and Yun Fu. Sparse subspace clustering by learning approximation ℓ0 codes. In Proc. of the AAAI Conf. on Artif. Intell., pages 2189-2195, 2017.","title":"Sparse subspace clustering by learning approximation ℓ0 codes","authors":"Li","year":"2017","venue":"Proc. of the AAAI Conf. on Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2017b] Jun Li, Tong Zhang, Wei Luo, Jian Yang, Xiaotong Yuan, and Jian Zhang. Sparseness analysis in the pertraining of deep neural networks. IEEE Trans. Neural Netw. Learn. Syst., 28(6):1425-1438, 2017.","title":"Sparseness analysis in the pertraining of deep neural networks","authors":"Li","year":"2017","venue":"IEEE Trans. Neural Netw. Learn. Syst","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2007] Ting Liu, Charles Rosenberg, and Henry A. Row- ley. Clustering billions of images with large scale nearest neighbor search. In Proc. of IEEE Workshop on App. of Comput. Vis., 2007.","title":"Rowley. Clustering billions of images with large scale nearest neighbor search","authors":"Ting Liu, Charles Rosenberg, Henry A","year":"2007","venue":"Proc. of IEEE Workshop on App. of Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2013] Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yi Ma. Robust recovery of subspace struc- tures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell., 35:171-184, 2013.","title":"Robust recovery of subspace structures by low-rank representation","authors":"Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, Yi Ma","year":"2013","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lu et al., 2012] Can-Yi Lu, Hai Min, Zhong-Qiu Zhao, Lin Zhu, De-Shuang Huang, and Shuicheng Yan. Robust and efficient subspace segmentation via least squares regression. In Proc. of Euro. Conf. Comput. Vis., pages 347-360, 2012.","title":"Robust and efficient subspace segmentation via least squares regression","authors":"Lu","year":"2012","venue":"Proc. of Euro. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nie et al., 2011] Feiping Nie, Zinan Zeng, Ivor W. Tsang, Dong Xu, and Changshui Zhang. Spectral embedded clustering: A framework for in-sample and out-of-sample spectral clustering. IEEE Trans. on Neural Netw., 22(11):1796-1808, 2011.","title":"Spectral embedded clustering: A framework for in-sample and out-of-sample spectral clustering","authors":"Feiping Nie, Zinan Zeng, Ivor W Tsang, Dong Xu, Changshui Zhang","year":"2011","venue":"IEEE Trans. on Neural Netw","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Peng et al., 2016] Xi Peng, Huajin Tang, Lei Zhang, Yi Zhang, and Shijie Xiao. A unified framework for representation-based subspace clustering of out-of-sample and large-scale data. IEEE Trans. Neural Netw. Learn. Syst., 27(12):2499-2512, 2016.","title":"A unified framework for representation-based subspace clustering of out-of-sample and large-scale data","authors":"Xi Peng, Huajin Tang, Lei Zhang, Yi Zhang, Shijie Xiao","year":"2016","venue":"IEEE Trans. Neural Netw. Learn. Syst","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shi and Malik, 2000] Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 22(8):888-905, 2000.","title":"Normalized cuts and image segmentation","authors":"Jianbo Shi, Jitendra Malik","year":"2000","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Talwalkar et al., 2013] Ameet Talwalkar, Lester Mackey, Yadong Mu, Shih-Fu Chang, and Michael I. Jordan. Distributed low-rank subspace segmentation. In Proc. of IEEE Int. Conf. Comput. Vis., pages 3543-3550, 2013.","title":"Distributed low-rank subspace segmentation","authors":"Ameet Talwalkar, Lester Mackey, Yadong Mu, Shih-Fu Chang, Michael I Jordan","year":"2013","venue":"Proc. of IEEE Int. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vidal, 2011] Rene Vidal. Subspace clustering. IEEE Signal Pro- cessing Magazine, 28(3):52-68, 2011.","title":"Subspace clustering","authors":"Rene Vidal","year":"2011","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Vincent et al., 2010] Pascal Vincent, Hugo Larochelle, Isabelle La- joie, Yoshua Bengio, and Pierre-Antoine Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res., 11:3371-3408, 2010.","title":"Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion","authors":"Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol","year":"2010","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2014] Shusen Wang, Bojun Tu, Congfu Xu, and Zhi- hua Zhang. Exact subspace clustering in linear time. In Proc. of the AAAI Conf. on Artif. Intell., pages 2113-2120, 2014.","title":"Exact subspace clustering in linear time","authors":"Wang","year":"2014","venue":"Proc. of the AAAI Conf. on Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xiao et al., 2014] Shijie Xiao, Mingkui Tan, and Dong Xu. Weight- ed block-sparse low rank representation for face clustering in videos. In Proc. of Euro. Conf. Comput. Vis., pages 123-138, 2014.","title":"Weighted block-sparse low rank representation for face clustering in videos","authors":"Shijie Xiao, Mingkui Tan, Dong Xu","year":"2014","venue":"Proc. of Euro. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yan et al., 2009] Donghui Yan, Ling Huang, and Michael I. Jordan. fast approximate spectral clustering. In Proc. of ACM SIGKDD Int. Conf. Knowl. Dis. and Data Min., pages 907-916, 2009.","title":"fast approximate spectral clustering","authors":"Donghui Yan, Ling Huang, Michael I Jordan","year":"2009","venue":"Proc. of ACM SIGKDD Int. Conf. Knowl. Dis. and Data Min","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yin et al., 2016] Ming Yin, Junbin Gao, and Zhouchen Lin. Lapla- cian regularized low-rank representation and its applications. IEEE Trans. Pattern Anal. Mach. Intell., 38(3):504-517, 2016.","title":"Laplacian regularized low-rank representation and its applications","authors":"Ming Yin, Junbin Gao, Zhouchen Lin","year":"2016","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2015] Changqing Zhang, Huazhu Fu, Si Liu, Guang- can Liu, and Xiaochun Cao. Low-rank tensor constrained mul- tiview subspace clustering. In Proc. of IEEE Int. Conf. Comput. Vis., pages 1582-1590, 2015.","title":"Low-rank tensor constrained multiview subspace clustering","authors":"Changqing Zhang, Huazhu Fu, Si Liu, Guangcan Liu, Xiaochun Cao","year":"2015","venue":"Proc. of IEEE Int. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhao and Fu, 2015a] Handong Zhao and Yun Fu. Dual-regularized multi-view outlier detection. In Proc. of of Int. Joint Conf. on Artif. Intell., pages 4077-4083, 2015.","title":"Dual-regularized multi-view outlier detection","authors":"Fu Zhao, Handong Zhao, Yun Fu","year":"2015","venue":"Proc. of of Int. Joint Conf. on Artif. Intell","doi":null,"url":null,"internal_url":"/proceedings/2015/572"},{"position":33,"raw":"[Zhao and Fu, 2015b] Handong Zhao and Yun Fu. Semantic single video segmentation with robust graph representation. In Proc. of of Int. Joint Conf. on Artif. Intell., pages 2219-2226, 2015.","title":"Semantic single video segmentation with robust graph representation","authors":"Fu Zhao, Handong Zhao, Yun Fu","year":"2015","venue":"Proc. of of Int. Joint Conf. on Artif. Intell","doi":null,"url":null,"internal_url":"/proceedings/2015/314"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}