{"type":"ScholarlyArticle","title":"Beyond the Nystrom Approximation: Speeding up Spectral Clustering using Uniform Sampling and Weighted Kernel k-means","authors":[{"name":"Mahesh Mohan","slug":"mahesh-mohan","given":"Mahesh","family":"Mohan","affiliation":"Washington University , Washington D.C."},{"name":"Claire Monteleoni","slug":"claire-monteleoni","given":"Claire","family":"Monteleoni","affiliation":"Washington University , Washington D.C."}],"abstract":"In this paper we present a framework for spectral clustering based on the following simple scheme: sample a subset of the input points, compute the clusters for the sampled subset using weighted kernel k-means (Dhillon et al. 2004) and use the resulting centers to compute a clustering for the remaining data points. For the case where the points are sampled uniformly at random without replacement, we show that the number of samples required depends mainly on the number of clusters and the diameter of the set of points in the kernel space. Experiments show that the proposed framework outperforms the approaches based on the Nystrom approximation both in terms of accuracy and computation time.","doi":"10.24963/ijcai.2017/347","url":"https://ijcai.org/proceedings/2017/347","pdf":"https://ijcai.org/proceedings/2017/0347.pdf","markdown":"https://ijcai.org/proceedings/2017/347.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/347","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":347,"pages":{"first":2494,"last":2500},"keywords":["Machine Learning: Data Mining","Machine Learning: Kernel Methods","Machine Learning: Unsupervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Bardenet et al., 2015] Rémi Bardenet, Odalric-Ambrym Maillard, et al. Concentration inequalities for sampling without replacement. Bernoulli, 21(3):1361-1385, 2015.","title":"Concentration inequalities for sampling without replacement","authors":"Bardenet","year":"2015","venue":"Bernoulli","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bouneffouf and Birol, 2015] Djallel Bouneffouf and Inanc Birol. Sampling with minimum sum of squared similari- ties for nystrom-based large scale spectral clustering. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence, 2015.","title":"Djallel Bouneffouf and Inanc Birol. Sampling with minimum sum of squared similarities for nystrom-based large scale spectral clustering","authors":"Birol Bouneffouf","year":"2015","venue":"Proceedings of the 23rd International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Butler, 2007] Steve Butler. Interlacing for weighted graphs using the normalized laplacian. Electronic Journal of Lin- ear Algebra, 16(1):8, 2007.","title":"Interlacing for weighted graphs using the normalized laplacian","authors":"Steve Butler","year":"2007","venue":"Electronic Journal of Linear Algebra","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Czumaj and Sohler, 2010] Artur Czumaj and Christian Sohler. Sublinear-time algorithms. In Property testing, pages 41-64. Springer, 2010.","title":"Artur Czumaj and Christian Sohler. Sublinear-time algorithms","authors":"Sohler Czumaj","year":"2010","venue":"Property testing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dhillon et al., 2004] Inderjit S Dhillon, Yuqiang Guan, and Brian Kulis. A unified view of kernel k-means, spectral clustering and graph cuts. Citeseer, 2004.","title":"A unified view of kernel k-means, spectral clustering and graph cuts","authors":"Yuqiang Inderjit S Dhillon, Brian Guan, Kulis","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gittens and Mahoney, 2013] Alex Gittens and Michael W Mahoney. Revisiting the nystrom method for im- proved large-scale machine learning. arXiv preprint arXiv:1303.1849, 2013.","title":"Revisiting the nystrom method for improved large-scale machine learning","authors":"Alex Gittens, Michael W Mahoney","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kumar et al., 2012] Sanjiv Kumar, Mehryar Mohri, and Ameet Talwalkar. Sampling methods for the nyström method. The Journal of Machine Learning Research, 13(1):981-1006, 2012.","title":"Sampling methods for the nyström method","authors":"Sanjiv Kumar, Mehryar Mohri, Ameet Talwalkar","year":"2012","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Serfling, 1974] Robert J Serfling. Probability inequalities for the sum in sampling without replacement. The Annals of Statistics, pages 39-48, 1974.","title":"Probability inequalities for the sum in sampling without replacement","authors":"J Robert, Serfling","year":"1974","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Talwalkar and Rostamizadeh, 2010] Ameet Talwalkar and Afshin Rostamizadeh. Matrix coherence and the nystrom method. arXiv preprint arXiv:1004.2008, 2010.","title":"Matrix coherence and the nystrom method","authors":"Ameet Talwalkar, Afshin Rostamizadeh","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Von Luxburg, 2007] Ulrike Von Luxburg. A tutorial on spectral clustering. Statistics and computing, 17(4):395- 416, 2007.","title":"A tutorial on spectral clustering","authors":"Ulrike Von, Luxburg","year":"2007","venue":"Statistics and computing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Williams and Seeger, 2001] Christopher Williams and Matthias Seeger. Using the nyström method to speed up kernel machines. In Proceedings of the 14th Annual Conference on Neural Information Processing Systems, number EPFL-CONF-161322, pages 682-688, 2001.","title":"Using the nyström method to speed up kernel machines","authors":"Christopher Williams, Matthias Seeger","year":"2001","venue":"Proceedings of the 14th Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Yan et al., 2009] Donghui Yan, Ling Huang, and Michael I Jordan. Fast approximate spectral clustering. In Proceed- ings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 907-916. ACM, 2009.","title":"Fast approximate spectral clustering","authors":"Donghui Yan, Ling Huang, Michael I Jordan","year":"2009","venue":"Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Yu and Shi, 2003] Stella X Yu and Jianbo Shi. Multiclass spectral clustering. In Computer Vision, 2003. Proceed- ings. Ninth IEEE International Conference on, pages 313- 319. IEEE, 2003.","title":"Multiclass spectral clustering","authors":"X Stella, Jianbo Yu, Shi","year":"2003","venue":"Proceedings. Ninth IEEE International Conference on","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}