{"type":"ScholarlyArticle","title":"Improved Algorithm on Online Clustering of Bandits","authors":[{"name":"Shuai Li","slug":"shuai-li","given":"Shuai","family":"Li","affiliation":"The Chinese University of Hong Kong; University of Cambridge"},{"name":"Wei Chen","slug":"wei-chen","given":"Wei","family":"Chen","affiliation":""},{"name":"Shuai Li","slug":"shuai-li","given":"Shuai","family":"Li","affiliation":"The Chinese University of Hong Kong; University of Cambridge"},{"name":"Kwong-Sak Leung","slug":"kwong-sak-leung","given":"Kwong-Sak","family":"Leung","affiliation":"The Chinese University of Hong Kong"}],"abstract":"We generalize the setting of online clustering of bandits by allowing non-uniform distribution over user frequencies. A more efficient algorithm is proposed with simple set structures to represent clusters. We prove a regret bound for the new algorithm which is free of the minimal frequency over users. The experiments on both synthetic and real datasets consistently show the advantage of the new algorithm over existing methods.","doi":"10.24963/ijcai.2019/405","url":"https://ijcai.org/proceedings/2019/405","pdf":"https://ijcai.org/proceedings/2019/0405.pdf","markdown":"https://ijcai.org/proceedings/2019/405.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/405","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":405,"pages":{"first":2923,"last":2929},"keywords":["Machine Learning: Reinforcement Learning","Machine Learning: Learning Theory","Machine Learning: Online Learning"],"videos":[],"references":[{"position":1,"raw":"[Abbasi-Yadkori et al., 2011] Yasin Abbasi-Yadkori, Dávid Pál, and Csaba Szepesvári. Improved algorithms for lin- ear stochastic bandits. In Advances in Neural Information Processing Systems, pages 2312-2320, 2011.","title":"Improved algorithms for linear stochastic bandits","authors":"Yasin Abbasi-Yadkori, Dávid Pál, Csaba Szepesvári","year":"2011","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Abe and Long, 1999] Naoki Abe and Philip M Long. As- sociative reinforcement learning using linear probabilistic concepts. In Proceedings of the Sixteenth International Conference on Machine Learning, pages 3-11.","title":"Associative reinforcement learning using linear probabilistic concepts","authors":"Naoki Abe, Philip M Long","year":null,"venue":"Proceedings of the Sixteenth International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Aggarwal, 2016] Charu Aggarwal. Recommender Systems. Springer, 2016.","title":"Recommender Systems","authors":"Charu Aggarwal","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bubeck et al., 2012] Sébastien Bubeck, Nicolo Cesa- Bianchi, et al. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends R in Machine Learning, 5(1):1-122, 2012.","title":"Regret analysis of stochastic and nonstochastic multi-armed bandit problems","authors":"Sébastien Bubeck, Nicolo Cesa-Bianchi","year":"2012","venue":"Foundations and Trends R in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Christakopoulou and Banerjee, 2018] Konstantina Chris- takopoulou and Arindam Banerjee. Learning to interact with users: A collaborative-bandit approach. In Proceed- ings of the 2018 SIAM International Conference on Data Mining, pages 612-620. SIAM, 2018.","title":"Learning to interact with users: A collaborative-bandit approach","authors":"Konstantina Christakopoulou, Arindam Banerjee","year":"2018","venue":"Proceedings of the 2018 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chu et al., 2011] Wei Chu, Lihong Li, Lev Reyzin, and Robert E Schapire. Contextual bandits with linear payoff functions. In AISTATS, volume 15, pages 208-214, 2011.","title":"Contextual bandits with linear payoff functions","authors":"Wei Chu, Lihong Li, Lev Reyzin, Robert E Schapire","year":"2011","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dani et al., 2008] Varsha Dani, Thomas P Hayes, and Sham M Kakade. Stochastic linear optimization under bandit feedback. In COLT, pages 355-366, 2008.","title":"Stochastic linear optimization under bandit feedback","authors":"Dani Varsha, Thomas P Hayes, M Sham, Kakade","year":"2008","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gentile et al., 2014] Claudio Gentile, Shuai Li, and Gio- vanni Zappella. Online clustering of bandits. In Pro- ceedings of the 31st International Conference on Machine Learning, pages 757-765, 2014.","title":"Online clustering of bandits","authors":"Claudio Gentile, Shuai Li, Giovanni Zappella","year":"2014","venue":"Proceedings of the 31st International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gentile et al., 2017] Claudio Gentile, Shuai Li, Purushot- tam Kar, Alexandros Karatzoglou, Giovanni Zappella, and Evans Etrue. On context-dependent clustering of bandits. In International Conference on Machine Learning, pages 1253-1262, 2017.","title":"On context-dependent clustering of bandits","authors":"Claudio Gentile, Shuai Li, Purushottam Kar, Alexandros Karatzoglou, Giovanni Zappella, Evans Etrue","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Harper and Konstan, 2016] F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. ACM Transactions on Interactive Intelligent Systems (TiiS), 5(4):19, 2016.","title":"The movielens datasets: History and context","authors":"Maxwell Harper, Joseph A Konstan","year":"2016","venue":"ACM Transactions on Interactive Intelligent Systems (TiiS)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Katariya et al., 2017a] Sumeet Katariya, Branislav Kveton, Csaba Szepesvári, Claire Vernade, and Zheng Wen. Bernoulli rank-1 bandits for click feedback. In Proceed- ings of the 26th International Joint Conference on Artifi- cial Intelligence, pages 2001-2007. AAAI Press, 2017.","title":"Bernoulli rank-1 bandits for click feedback","authors":"Katariya","year":"2017","venue":"Proceedings of the 26th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/278"},{"position":12,"raw":"[Katariya et al., 2017b] Sumeet Katariya, Branislav Kve- ton, Csaba Szepesvari, Claire Vernade, and Zheng Wen. Stochastic rank-1 bandits. In Artificial Intelligence and Statistics, pages 392-401, 2017.","title":"Stochastic rank-1 bandits","authors":"Katariya","year":"2017","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Korda et al., 2016] Nathan Korda, Balazs Szorenyi, and Shuai Li. Distributed clustering of linear bandits in peer to peer networks. In The 33rd International Conference on Machine Learning (ICML), 2016. [Kveton et al., 2017] Branislav Kveton, Csaba Szepesvari, Anup Rao, Zheng Wen, Yasin Abbasi-Yadkori, and S Muthukrishnan. Stochastic low-rank bandits. arXiv preprint arXiv:1712.04644, 2017.","title":"Distributed clustering of linear bandits in peer to peer networks","authors":"Nathan Korda, Balazs Szorenyi, Shuai Li ; Kveton","year":"2016","venue":"The 33rd International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kwon, 2018] Jin Kyoung Kwon. Overlapping clustering of contextual bandits with nmf techniques. 2018.","title":"Overlapping clustering of contextual bandits with nmf techniques","authors":"Jin Kyoung, Kwon","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lattimore and Szepesvári, 2018] Tor Lattimore and Csaba Szepesvári. Bandit Algorithms. preprint, 2018.","title":"Tor Lattimore and Csaba Szepesvári","authors":"Szepesvári Lattimore","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li and Zhang, 2018] Shuai Li and Shengyu Zhang. On- line clustering of contextual cascading bandits. In Thirty- Second AAAI Conference on Artificial Intelligence, 2018.","title":"Online clustering of contextual cascading bandits","authors":"Li, Zhang, Shuai Li, Shengyu Zhang","year":"2018","venue":"Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2010] Lihong Li, Wei Chu, John Langford, and Robert E Schapire. A contextual-bandit approach to per- sonalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pages 661-670. ACM, 2010.","title":"A contextual-bandit approach to personalized news article recommendation","authors":"Lihong Li, Wei Chu, John Langford, Robert E Schapire","year":"2010","venue":"Proceedings of the 19th international conference on World wide web","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2016] Shuai Li, Alexandros Karatzoglou, and Claudio Gentile. Collaborative filtering bandits. In Pro- ceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pages 539-548. ACM, 2016.","title":"Collaborative filtering bandits","authors":"Shuai Li, Alexandros Karatzoglou, Claudio Gentile","year":"2016","venue":"Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li, 2016] Shuai Li. The art of clustering bandits. PhD the- sis, Università degli Studi dell'Insubria, 2016.","title":"The art of clustering bandits","authors":"Shuai Li","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Linden et al., 2003] Greg Linden, Brent Smith, and Jeremy York. Amazon. com recommendations: Item-to-item col- laborative filtering. IEEE Internet computing, 7(1):76-80, 2003.","title":"Amazon. com recommendations: Item-to-item collaborative filtering","authors":"Greg Linden, Brent Smith, Jeremy York","year":"2003","venue":"IEEE Internet computing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[MacQueen, 1967] James MacQueen. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathemat- ical statistics and probability, volume 1, pages 281-297.","title":"Some methods for classification and analysis of multivariate observations","authors":"James Macqueen","year":null,"venue":"Proceedings of the fifth Berkeley symposium on mathematical statistics and probability","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Nguyen and Lauw, 2014] Trong T Nguyen and Hady W Lauw. Dynamic clustering of contextual multi-armed ban- dits. In Proceedings of the 23rd ACM International Con- ference on Conference on Information and Knowledge Management, pages 1959-1962. ACM, 2014.","title":"Dynamic clustering of contextual multi-armed bandits","authors":"T Trong, Hady W Nguyen, Lauw","year":"2014","venue":"Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rusmevichientong and Tsitsiklis, 2010] Paat Rus- mevichientong and John N Tsitsiklis. Linearly param- eterized bandits. Mathematics of Operations Research, 35(2):395-411, 2010.","title":"Linearly parameterized bandits","authors":"Paat Rusmevichientong, John N Tsitsiklis","year":"2010","venue":"Mathematics of Operations Research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Turgay et al., 2018] Eralp Turgay, Doruk Oner, and Cem Tekin. Multi-objective contextual bandit problem with similarity information. In International Conference on Ar- tificial Intelligence and Statistics, pages 1673-1681, 2018.","title":"Multi-objective contextual bandit problem with similarity information","authors":"Eralp Turgay, Doruk Oner, Cem Tekin","year":"2018","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Woo et al., 2014] Choong-Wan Woo, Anjali Krishnan, and Tor D Wager. Cluster-extent based thresholding in fmri analyses: pitfalls and recommendations. Neuroimage, 91:412-419, 2014.","title":"Cluster-extent based thresholding in fmri analyses: pitfalls and recommendations","authors":"Choong-Wan Woo, Anjali Krishnan, Tor D Wager","year":"2014","venue":"Neuroimage","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}