{"type":"ScholarlyArticle","title":"Projective Low-rank Subspace Clustering via Learning Deep Encoder","authors":[{"name":"Jun Li","slug":"jun-li","given":"Jun","family":"Li","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA , 02115 , USA."},{"name":"Liu Hongfu","slug":"liu-hongfu","given":"Liu","family":"Hongfu","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA , 02115 , USA."},{"name":"Handong Zhao","slug":"handong-zhao","given":"Handong","family":"Zhao","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA , 02115 , USA."},{"name":"Yun Fu","slug":"yun-fu","given":"Yun","family":"Fu","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA , 02115 , USA.; College of Computer and Information Science , Northeastern University , Boston , MA , 02115 , USA."}],"abstract":"Low-rank subspace clustering (LRSC) has been considered as the state-of-the-art method on small datasets. LRSC constructs a desired similarity graph by low-rank representation (LRR), and employs a spectral clustering to segment the data samples. However, effectively applying LRSC into clustering big data becomes a challenge because both LRR and spectral clustering suffer from high computational cost. To address this challenge, we create a projective low-rank subspace clustering (PLrSC) scheme for large scale clustering problem. First, a small dataset is randomly sampled from big dataset. Second, our proposed predictive low-rank decomposition (PLD) is applied to train a deep encoder by using the small dataset, and the deep encoder is used to fast compute the low-rank representations of all data samples. Third, fast spectral clustering is employed to segment the representations. As a non-trivial contribution, we theoretically prove the deep encoder can universally approximate to the exact (or bounded) recovery of the row space. Experiments verify that our scheme outperforms the related methods on large scale datasets in a small amount of time. We achieve the state-of-art clustering accuracy by 95.8% on MNIST using scattering convolution features.","doi":"10.24963/ijcai.2017/298","url":"https://ijcai.org/proceedings/2017/298","pdf":"https://ijcai.org/proceedings/2017/0298.pdf","markdown":"https://ijcai.org/proceedings/2017/298.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/298","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":298,"pages":{"first":2145,"last":2151},"keywords":["Machine Learning: Machine Learning","Robotics and Vision: Vision and Perception"],"videos":[],"references":[{"position":1,"raw":"[Bengio et al., 2013] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell., 35(8):1798-1828, Aug. 2013.","title":"Representation learning: A review and new perspectives","authors":"Yoshua Bengio, Aaron Courville, Pascal Vincent","year":"2013","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bruna and Mallat, 2013] Joan Bruna and Stephane Mallat. Invari- ant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell., 35(8):1872-1886, Aug. 2013.","title":"Invariant scattering convolution networks","authors":"Bruna, Mallat, Joan Bruna, Stephane Mallat","year":"2013","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai and Chen, 2015] Deng Cai and Xinlei Chen. Large scale spec- tral clustering via landmark-based sparse representation. IEEE Trans. on Cybernetics, 45(8):1669-1680, 2015.","title":"Large scale spectral clustering via landmark-based sparse representation","authors":"Chen ; ] Deng Cai, Xinlei Cai, Chen","year":"2015","venue":"IEEE Trans. on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai et al., 2010] Jian-Feng Cai, Emmanuel J. Candés, and Zuowei Shen. A singular value thresholding algorithm for matrix comple- tion. SIAM J. Optim., 20(4):1956-1982, Mar. 2010.","title":"A singular value thresholding algorithm for matrix completion","authors":"Jian-Feng Cai, Emmanuel J Candés, Zuowei Shen","year":"2010","venue":"SIAM J. Optim","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Candés et al., 2011] Emmanuel J. Candés, Xiaodong Li, Yi Ma, and John Wright. Robust principal component analysis? Journal of the ACM, 58:1-37, 2011.","title":"Robust principal component analysis","authors":"Emmanuel J Candés, Xiaodong Li, Yi Ma, John Wright","year":"2011","venue":"Journal of the ACM","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2011] Wen-Yen Chen, Yangqiu Song, Hongjie Bai, Chih-Jen Lin, and Edward Y. Chang. Parallel spectral clustering in distributed systems. IEEE Trans. Pattern Anal. Mach. Intell., 33(3):568-586, 2011.","title":"Parallel spectral clustering in distributed systems","authors":"Wen-Yen Chen, Yangqiu Song, Hongjie Bai, Chih-Jen Lin, Edward Y Chang","year":"2011","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ding et al., 2015] Zhengming Ding, Ming Shao, and Yun Fu. Deep low-rank coding for transfer learning. In Proc. of of Int. Joint Conf. on Artif. Intell., pages 3453-3459, 2015.","title":"Deep low-rank coding for transfer learning","authors":"Zhengming Ding, Ming Shao, Yun Fu","year":"2015","venue":"Proc. of of Int. Joint Conf. on Artif. Intell","doi":null,"url":null,"internal_url":"/proceedings/2015/486"},{"position":8,"raw":"[Ding et al., 2016] Zhengming Ding, Ming Shao, and Yun Fu. Deep robust encoder through locality preserving low-rank dictionary. In Proc. of Euro. Conf. Comput. Vis., pages 567-582, 2016.","title":"Deep robust encoder through locality preserving low-rank dictionary","authors":"Zhengming Ding, Ming Shao, Yun Fu","year":"2016","venue":"Proc. of Euro. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Elhamifar and Vidal, 2013] Ehsan Elhamifar and Rene Vidal. S- parse subspace clustering: Algorithm, theory, and applications. IEEE Trans. Pattern Anal. Mach. Intell., 35:2765-2781, 2013.","title":"Sparse subspace clustering: Algorithm, theory, and applications","authors":"Ehsan Elhamifar, Rene Vidal","year":"2013","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fowlkes et al., 2004] Charless Fowlkes, Serge Belongie, Fan Chung, and Jitendra Malik. Spectral grouping using the nystrom method. IEEE Trans. Pattern Anal. Mach. Intell., 26(2):214-225, 2004.","title":"Spectral grouping using the nystrom method","authors":"Charless Fowlkes, Serge Belongie, Fan Chung, Jitendra Malik","year":"2004","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gregor and LeCun, 2010] Karol Gregor and Yann LeCun. Learning fast approximations of sparse coding. In Proc. of Int. Conf. Mach. Learn., pages 399-406, 2010.","title":"Learning fast approximations of sparse coding","authors":"Karol Gregor, Yann Lecun","year":"2010","venue":"Proc. of Int. Conf. Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Haykin, 2009] Simon Haykin. In Neural Networks and Learning Machines. Pearson Education Inc., 2009.","title":"Neural Networks and Learning Machines","authors":"Simon Haykin","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li and Fu, 2015] Sheng Li and Yun Fu. Learning balanced and unbalanced graphs via low-rank coding. IEEE Trans. on Knowl. Data Eng., 27:1274-1287, 2015.","title":"Learning balanced and unbalanced graphs via low-rank coding","authors":"Sheng Li, Yun Fu","year":"2015","venue":"IEEE Trans. on Knowl. Data Eng","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2014] Jun Li, Heyou Chang, and Jian Yang. Learning discriminative low-rank representations for image classification. In Proc. of Int. Joint Conf. Neural Netw., pages 313-318, 2014.","title":"Learning discriminative low-rank representations for image classification","authors":"Li","year":"2014","venue":"Proc. of Int. Joint Conf. Neural Netw","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2015] Jun Li, Heyou Chang, and Jian Yang. Sparse deep stacking network for image classification. In Proc. of the AAAI Conf. on Artif. Intell., pages 3804-3810, 2015.","title":"Sparse deep stacking network for image classification","authors":"Li","year":"2015","venue":"Proc. of the AAAI Conf. on Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2016] Jun Li, Yu Kong, Handong Zhao, Jian Yang, and Yun Fu. Learning fast low-rank projection for image classification. IEEE Trans. Image Process., 25(10):4803-4814, 2016.","title":"Learning fast low-rank projection for image classification","authors":"Li","year":"2016","venue":"IEEE Trans. Image Process","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2017a] Jun Li, Yu Kong, and Yun Fu. Sparse subspace clustering by learning approximation ℓ0 codes. In Proc. of the AAAI Conf. on Artif. Intell., pages 2189-2195, 2017.","title":"Sparse subspace clustering by learning approximation ℓ0 codes","authors":"Li","year":"2017","venue":"Proc. of the AAAI Conf. on Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2017b] Jun Li, Tong Zhang, Wei Luo, Jian Yang, Xiaotong Yuan, and Jian Zhang. Sparseness analysis in the pertraining of deep neural networks. IEEE Trans. Neural Netw. Learn. Syst., 28(6):1425-1438, 2017.","title":"Sparseness analysis in the pertraining of deep neural networks","authors":"Li","year":"2017","venue":"IEEE Trans. Neural Netw. Learn. Syst","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lin et al., 2011] Zhouchen Lin, Risheng Liu, and Zhixun Su. Lin- earized alternating direction method with adaptive penality for low rank representation. In Proc. of Adv. Neural Inf. Process. Syst., pages 612-620, 2011.","title":"Linearized alternating direction method with adaptive penality for low rank representation","authors":"Lin","year":"2011","venue":"Proc. of Adv. Neural Inf. Process. Syst","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu and Yan, 2011] Guangcan Liu and Shuicheng Yan. Latent low- rank representation for subspace segmentation and feature extrac- tion. In Proc. of IEEE Int. Conf. Comput. Vis., pages 1615-1622, 2011.","title":"Latent lowrank representation for subspace segmentation and feature extraction","authors":"Yan ; Liu, Guangcan Liu, Shuicheng Yan","year":"2011","venue":"Proc. of IEEE Int. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2013] Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yi Ma. Robust recovery of subspace struc- tures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell., 35:171-184, 2013.","title":"Robust recovery of subspace structures by low-rank representation","authors":"Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, Yi Ma","year":"2013","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Peng et al., 2016] Xi Peng, Huajin Tang, Lei Zhang, Yi Zhang, and Shijie Xiao. A unified framework for representation-based subspace clustering of out-of-sample and large-scale data. IEEE Trans. Neural Netw. Learn. Syst., 27(12):2499-2512, 2016.","title":"A unified framework for representation-based subspace clustering of out-of-sample and large-scale data","authors":"Xi Peng, Huajin Tang, Lei Zhang, Yi Zhang, Shijie Xiao","year":"2016","venue":"IEEE Trans. Neural Netw. Learn. Syst","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ripley, 1996] Brian D. Ripley. In Pattern Recognition and Neural Networks. Cambridge University Press, 1996.","title":"Pattern Recognition and Neural Networks","authors":"Brian D Ripley","year":"1996","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shen et al., 2016] Jie Shen, Ping Li, and Huan Xu. Online low-rank subspace clustering by basis dictionary pursuit. In Proc. of Int. Conf. Mach. Learn., 2016.","title":"Online low-rank subspace clustering by basis dictionary pursuit","authors":"Jie Shen, Ping Li, Huan Xu","year":"2016","venue":"Proc. of Int. Conf. Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Shi and Malik, 2000] Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 22(8):888-905, 2000.","title":"Normalized cuts and image segmentation","authors":"Jianbo Shi, Jitendra Malik","year":"2000","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sprechmann et al., 2015] Pablo Sprechmann, Alex M. Bronstein, and Guillermo Sapiro. Learning efficient sparse and low rank models. IEEE Trans. Pattern Anal. Mach. Intell., 37(9):1821- 1833, 2015.","title":"Learning efficient sparse and low rank models","authors":"Sprechmann","year":"2015","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Talwalkar et al., 2013] Ameet Talwalkar, Lester Mackey, Yadong Mu, Shih-Fu Chang, and Michael I. Jordan. Distributed low-rank subspace segmentation. In Proc. of IEEE Int. Conf. Comput. Vis., pages 3543-3550, 2013.","title":"Distributed low-rank subspace segmentation","authors":"Ameet Talwalkar, Lester Mackey, Yadong Mu, Shih-Fu Chang, Michael I Jordan","year":"2013","venue":"Proc. of IEEE Int. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Vincent et al., 2010] Pascal Vincent, Hugo Larochelle, Isabelle La- joie, Yoshua Bengio, and Pierre-Antoine Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res., 11:3371-3408, 2010.","title":"Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion","authors":"Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol","year":"2010","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2014] Shusen Wang, Bojun Tu, Congfu Xu, and Zhi- hua Zhang. Exact subspace clustering in linear time. In Proc. of the AAAI Conf. on Artif. Intell., pages 2113-2120, 2014.","title":"Exact subspace clustering in linear time","authors":"Wang","year":"2014","venue":"Proc. of the AAAI Conf. on Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2015] Yu Wang, David Wipf, Qing Ling, Wei Chen, and Ian Wassell. Multi-task learning for subspace segmentation. In Proc. of Int. Conf. Mach. Learn., 2015.","title":"Multi-task learning for subspace segmentation","authors":"Yu Wang, David Wipf, Qing Ling, Wei Chen, Ian Wassell","year":"2015","venue":"Proc. of Int. Conf. Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2009] Junfeng Yang, Wotao Yin, Yin Zhang, and Yilun Wang. A fast algorithm for edge-preserving variational multi- channel image restoration. SIAM J. Imaging Sciences, 2:569-592, 2009.","title":"A fast algorithm for edge-preserving variational multichannel image restoration","authors":"Junfeng Yang, Wotao Yin, Yin Zhang, Yilun Wang","year":"2009","venue":"SIAM J. Imaging Sciences","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[You et al., 2016] Chong You, Chun-Guang Li, Daniel Robinson, and Rene Vidal. Oracle based active set algorithm for scalable elastic net subspace clustering. In Proc. of IEEE Comput. Vis. Pattern Recognit., pages 3928-3937, 2016.","title":"Oracle based active set algorithm for scalable elastic net subspace clustering","authors":"Chong You, Chun-Guang Li, Daniel Robinson, Rene Vidal","year":"2016","venue":"Proc. of IEEE Comput. Vis. Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2015] Changqing Zhang, Huazhu Fu, Si Liu, Guang- can Liu, and Xiaochun Cao. Low-rank tensor constrained mul- tiview subspace clustering. In Proc. of IEEE Int. Conf. Comput. Vis., pages 1582-1590, 2015.","title":"Low-rank tensor constrained multiview subspace clustering","authors":"Changqing Zhang, Huazhu Fu, Si Liu, Guangcan Liu, Xiaochun Cao","year":"2015","venue":"Proc. of IEEE Int. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhao and Fu, 2015] Handong Zhao and Yun Fu. Semantic single video segmentation with robust graph representation. In Proc. of of Int. Joint Conf. on Artif. Intell., pages 2219-2226, 2015.","title":"Semantic single video segmentation with robust graph representation","authors":"Handong Zhao, Yun Fu","year":"2015","venue":"Proc. of of Int. Joint Conf. on Artif. Intell","doi":null,"url":null,"internal_url":"/proceedings/2015/314"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}