{"type":"ScholarlyArticle","title":"FLS: A New Local Search Algorithm for K-means with Smaller Search Space","authors":[{"name":"Junyu Huang","slug":"junyu-huang","given":"Junyu","family":"Huang","affiliation":"School of Computer Science and Engineering , Central South University , Changsha 410083 , China"},{"name":"Qilong Feng","slug":"qilong-feng","given":"Qilong","family":"Feng","affiliation":"School of Computer Science and Engineering , Central South University , Changsha 410083 , China"},{"name":"Ziyun Huang","slug":"ziyun-huang","given":"Ziyun","family":"Huang","affiliation":"Department of Computer Science and Software Engineering , Penn State Erie , the Behrend College"},{"name":"Jinhui Xu","slug":"jinhui-xu","given":"Jinhui","family":"Xu","affiliation":"Department of Computer Science and Engineering , State University of New York at Buffalo , NY , USA"},{"name":"Jianxin Wang","slug":"jianxin-wang","given":"Jianxin","family":"Wang","affiliation":"School of Computer Science and Engineering , Central South University , Changsha 410083 , China; The Hunan Provincial Key Lab of Bioinformatics , Central South University , Changsha 410083 , China"}],"abstract":"The k-means problem is an extensively studied unsupervised learning problem with various applications in decision making and data mining. In this paper, we propose a fast and practical local search algorithm for the k-means problem. Our method reduces the search space of swap pairs from O(nk) to O(k^2), and applies random mutations to find potentially better solutions when local search falls into poor local optimum. With the assumption of data distribution that each optimal cluster has \"average\" size of \\Omega(n/k), which is common in many datasets and k-means benchmarks, we prove that our proposed algorithm gives a (100+\\epsilon)-approximate solution in expectation. Empirical experiments show that our algorithm achieves better performance compared to existing state-of-the-art local search methods on k-means benchmarks and large datasets.","doi":"10.24963/ijcai.2022/429","url":"https://ijcai.org/proceedings/2022/429","pdf":"https://ijcai.org/proceedings/2022/0429.pdf","markdown":"https://ijcai.org/proceedings/2022/429.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/429","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":429,"pages":{"first":3092,"last":3098},"keywords":["Machine Learning: Clustering","Search: Heuristic Search","Search: Combinatorial Search and Optimisation","Search: Local search","Search: Applications"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986721","length_sec":81,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984864","length_sec":297,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Angel et al., 2015] Eric Angel, Nguyen Kim Thang, and Damien Regnault. Improved local search for universal facility location. Journal of Combinatorial Optimization, 29:237-246, 2015.","title":"Improved local search for universal facility location","authors":"Eric Angel, Nguyen Kim Thang, Damien Regnault","year":"2015","venue":"Journal of Combinatorial Optimization","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arthur and Vassilvitskii, 2007] David Arthur and Sergei Vassilvitskii. k-means++: The advantages of careful seed- ing. In Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1027-1035.","title":"k-means++: The advantages of careful seeding","authors":"David Arthur, Sergei Vassilvitskii","year":null,"venue":"Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Arya et al., 2004] Vijay Arya, Naveen Garg, Rohit Khan- dekar, Adam Meyerson, Kamesh Munagala, and Vinayaka Pandit. Local search heuristics for k-median and facility location problems. SIAM Journal on Computing, 33:544- 562, 2004.","title":"Local search heuristics for k-median and facility location problems","authors":"Vijay Arya, Naveen Garg, Rohit Khandekar, Adam Meyerson, Kamesh Munagala, Vinayaka Pandit","year":"2004","venue":"SIAM Journal on Computing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blum et al., 1973] Manuel Blum, Robert W. Floyd, Vaughan R. Pratt, Ronald L. Rivest, and Robert Endre Tarjan. Time bounds for selection. Journal of Computer and System Sciences, 7:448-461, 1973.","title":"Time bounds for selection","authors":"Manuel Blum, Robert W Floyd, Vaughan R Pratt, Ronald L Rivest, Robert Endre Tarjan","year":"1973","venue":"Journal of Computer and System Sciences","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Choo et al., 2020] Davin Choo, Christoph Grunau, Julian Portmann, and Václav Rozhon. k-means++: Few more steps yield constant approximation. In International Con- ference on Machine Learning, pages 1909-1917. PMLR, 2020.","title":"k-means++: Few more steps yield constant approximation","authors":"Davin Choo, Christoph Grunau, Julian Portmann, Václav Rozhon","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ding and Huang, 2021] Hu Ding and Jiawei Huang. Is simple uniform sampling efficient for center-based clus- tering with outliers: when and why? arXiv preprint arXiv:2103.00558, 2021.","title":"Is simple uniform sampling efficient for center-based clustering with outliers: when and why","authors":"Huang ; ] Ding, Ding Hu, Jiawei Huang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fränti and Sieranoja, 2018] Pasi Fränti and Sami Sieranoja. K-means properties on six clustering benchmark datasets. Applied Intelligence, 48:4743-4759, 2018.","title":"Pasi Fränti and Sami Sieranoja. K-means properties on six clustering benchmark datasets","authors":"Sieranoja Fränti","year":"2018","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Friggstad et al., 2019] Zachary Friggstad, Mohsen Reza- pour, and Mohammad R Salavatipour. Local search yields PTAS for k-means in doubling metrics. SIAM Journal on Computing, 48:452-480, 2019.","title":"Local search yields PTAS for k-means in doubling metrics","authors":"Zachary Friggstad, Mohsen Rezapour, Mohammad R Salavatipour","year":"2019","venue":"SIAM Journal on Computing","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gupta and Tangwongsan, 2008] Anupam Gupta and Kanat Tangwongsan. Simpler analyses of local search algorithms for facility location. arXiv preprint arXiv:0809.2554, 2008.","title":"Simpler analyses of local search algorithms for facility location","authors":"Anupam Gupta, Kanat Tangwongsan","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gupta et al., 2017] Shalmoli Gupta, Ravi Kumar, Kefu Lu, Benjamin Moseley, and Sergei Vassilvitskii. Local search methods for k-means with outliers. Proceedings of the VLDB Endowment, 10:757-768, 2017.","title":"Local search methods for k-means with outliers","authors":"Gupta","year":"2017","venue":"Proceedings of the VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kanungo et al., 2004] Tapas Kanungo, David M Mount, Nathan S Netanyahu, Christine D Piatko, Ruth Silverman, and Angela Y Wu. A local search approximation algo- rithm for k-means clustering. Computational Geometry, 28:89-112, 2004.","title":"A local search approximation algorithm for k-means clustering","authors":"Kanungo","year":"2004","venue":"Computational Geometry","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lattanzi and Sohler, 2019] Silvio Lattanzi and Christian Sohler. A better k-means++ algorithm via local search. In International Conference on Machine Learning, pages 3662-3671. PMLR, 2019.","title":"A better k-means++ algorithm via local search","authors":"Silvio Lattanzi, Christian Sohler","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lloyd, 1982] Stuart Lloyd. Least squares quantization in PCM. IEEE Transactions on Information Theory, 28:129- 137, 1982.","title":"Least squares quantization in PCM","authors":"Stuart Lloyd","year":"1982","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mahabadi and Vakilian, 2020] Sepideh Mahabadi and Ali Vakilian. Individual fairness for k-clustering. In Inter- national Conference on Machine Learning, pages 6586- 6596. PMLR, 2020.","title":"Sepideh Mahabadi and Ali Vakilian. Individual fairness for k-clustering","authors":"Vakilian Mahabadi","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Meyerson et al., 2004] Adam Meyerson, Liadan O'callaghan, and Serge Plotkin. A k-median algo- rithm with running time independent of data size. Machine Learning, 56:61-87, 2004.","title":"A k-median algorithm with running time independent of data size","authors":"Adam Meyerson, O Liadan, Serge 'callaghan, Plotkin","year":"2004","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rozhoň, 2020] Václav Rozhoň. Simple and sharp analy- sis of k-means||. In International Conference on Machine Learning, pages 8266-8275. PMLR, 2020.","title":"Václav Rozhoň. Simple and sharp analysis of k","authors":"Rozhoň","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Zhang et al., 2019] Dongmei Zhang, Chunlin Hao, Chenchen Wu, Dachuan Xu, and Zhenning Zhang. Local search approximation algorithms for the k-means problem with penalties. Journal of Combinatorial Optimization, 37:439-453, 2019.","title":"Local search approximation algorithms for the k-means problem with penalties","authors":"Dongmei Zhang, Chunlin Hao, Chenchen Wu, Dachuan Xu, Zhenning Zhang","year":"2019","venue":"Journal of Combinatorial Optimization","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}