{"type":"ScholarlyArticle","title":"Combinatorial Multi-Armed Bandits with Concave Rewards and Fairness Constraints","authors":[{"name":"Huanle Xu","slug":"huanle-xu","given":"Huanle","family":"Xu","affiliation":"Dongguan University of Technology ,"},{"name":"Yang Liu","slug":"yang-liu","given":"Yang","family":"Liu","affiliation":"The Chinese University of Hong Kong"},{"name":"Wing Cheong Lau","slug":"wing-cheong-lau","given":"Wing Cheong","family":"Lau","affiliation":"The Chinese University of Hong Kong"},{"name":"Rui Li","slug":"rui-li","given":"Rui","family":"Li","affiliation":"Dongguan University of Technology ,"}],"abstract":"The problem of multi-armed bandit (MAB) with fairness constraint has emerged as an important research topic recently. For such problems, one common objective is to maximize the total rewards within a fixed round of pulls, while satisfying the fairness requirement of a minimum selection fraction for each individual arm in the long run. Previous works have made substantial advancements in designing efficient online selection solutions, however, they fail to achieve a sublinear regret bound when incorporating such fairness constraints. In this paper, we study a combinatorial MAB problem with concave objective and fairness constraints. In particular, we adopt a new approach that combines online convex optimization with bandit methods to design selection algorithms. Our algorithm is computationally efficient, and more importantly, manages to achieve a sublinear regret bound with probability guarantees. Finally, we evaluate the performance of our algorithm via extensive simulations and demonstrate that it outperforms the baselines substantially.","doi":"10.24963/ijcai.2020/354","url":"https://ijcai.org/proceedings/2020/354","pdf":"https://ijcai.org/proceedings/2020/0354.pdf","markdown":"https://ijcai.org/proceedings/2020/354.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/354","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":354,"pages":{"first":2554,"last":2560},"keywords":["Machine Learning: Online Learning","Uncertainty in AI: Other"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/26785/file_26785.m3u8","length_sec":745,"provider":"hls"}],"references":[{"position":1,"raw":"[Agrawal and Devanur, 2015] Shipra Agrawal and Nikhil R. Devanur. Fast algorithms for online stochastic convex pro- gramming. In Proceedings of SODA, 2015.","title":"Fast algorithms for online stochastic convex programming","authors":"Shipra Agrawal, Nikhil R Devanur","year":"2015","venue":"Proceedings of SODA","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Agrawal and Devanur, 2016] Shipra Agrawal and Nikhil R. Devanur. Linear contextual bandits with knapsacks. In Proceedings of NIPS, 2016.","title":"Linear contextual bandits with knapsacks","authors":"Shipra Agrawal, Nikhil R Devanur","year":"2016","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Agrawal and Devanurr, 2014] Shipra Agrawal and Nikhil R. Devanurr. Bandits with concave rewards and convex knapsacks. In ACM Conference on Economics & Computation, 2014.","title":"Bandits with concave rewards and convex knapsacks","authors":"Shipra Agrawal, Nikhil R Devanurr","year":"2014","venue":"ACM Conference on Economics & Computation","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Auer et al., 2002] Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47, 2002.","title":"Finite-time analysis of the multiarmed bandit problem","authors":"Auer","year":"2002","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Auer, 2002] Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. In Journal of Machine Learning Research, 2002.","title":"Using confidence bounds for exploitation-exploration trade-offs","authors":"Peter Auer","year":"2002","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Badanidiyuru et al., 2013] Ashwinkumar Badanidiyuru, Robert Kleinberg, and Aleksandrs Slivkins. Bandits with knapsacks. In IEEE Symposium on Foundations of Computer Science (FOCS), 2013.","title":"Ashwinkumar Badanidiyuru, Robert Kleinberg, and Aleksandrs Slivkins. Bandits with knapsacks","authors":"Badanidiyuru","year":"2013","venue":"IEEE Symposium on Foundations of Computer Science (FOCS)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Badanidiyuru et al., 2018] Ashwinkumar Badanidiyuru, Robert Kleinberg, and Aleksandrs Slivkins. Bandits with knapsacks. In Journal of the ACM, 2018.","title":"Ashwinkumar Badanidiyuru, Robert Kleinberg, and Aleksandrs Slivkins. Bandits with knapsacks","authors":"Badanidiyuru","year":"2018","venue":"Journal of the ACM","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cavalcante and Stanczak, 2018] Renato Luis Garrido Cav- alcante and Slawomir Stanczak. Fundamental properties of solutions to utility maximization problems in wireless networks. In arXiv:1610.01988, 2018.","title":"Fundamental properties of solutions to utility maximization problems in wireless networks","authors":"Stanczak Cavalcante","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chen et al., 2013] Wei Chen, Yajun Wang, and Yang Yuanu. Combinatorial multi-armed bandit: General framework, results and applications. In Proceeding of ICML, 2013.","title":"Combinatorial multi-armed bandit: General framework, results and applications","authors":"Wei Chen, Yajun Wang, Yang Yuanu","year":"2013","venue":"Proceeding of ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Chen et al., 2016] Wei Chen, Wei Hu, Fu Li, Jian Li, Yu Liu, and Pinyan Lu. Combinatorial multi-armed bandit with general reward functions. In Proceedings of NIPS, 2016.","title":"Combinatorial multi-armed bandit with general reward functions","authors":"Wei Chen, Wei Hu, Fu Li, Jian Li, Yu Liu, Pinyan Lu","year":"2016","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Combes et al., 2015] Richard Combes, M. Sadegh Talebi, Alexandre Proutiere, and Marc Lelarge. Combinatorial bandits revisited. In Proceedings of NIPS, 2015.","title":"Combinatorial bandits revisited","authors":"M Sadegh Richard Combes, Alexandre Talebi, Marc Proutiere, Lelarge","year":"2015","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Dai et al., 2016] Haipeng Dai, Yunhuai Liu, Alex X. Liu, Lingtao Kong, Guihai Chen, and Tian He. Radiation con- strained wireless charger placement. In Proceedings of In- focom, 2016.","title":"Radiation constrained wireless charger placement","authors":"Haipeng Dai, Yunhuai Liu, Alex X Liu, Lingtao Kong, Guihai Chen, Tian He","year":"2016","venue":"Proceedings of Infocom","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Dickerson et al., 2019] John P. Dickerson, Karthik Abinav Sankararaman, Kanthi Kiran Sarpatwar, Aravind Srini- vasan, Kun-Lung Wu, and Pan Xu. Online resource alloca- tion with matching constraints. In Proceedings of Interna- tional Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2019.","title":"Online resource allocation with matching constraints","authors":"John P Dickerson, Karthik Abinav Sankararaman, Kanthi Kiran Sarpatwar, Aravind Srinivasan, Kun-Lung Wu, Pan Xu","year":"2019","venue":"Proceedings of International Conference on Autonomous Agents and Multiagent Systems (AAMAS)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ferdosian and et al., 2018] Nasim Ferdosian and Mo- hamed Othman et al. Fair-QoS broker algorithm for overload-state downlink resource scheduling in lte networks. IEEE SYSTEMS JOURNAL, 2018.","title":"Fair-QoS broker algorithm for overload-state downlink resource scheduling in lte networks","authors":"Nasim Ferdosian, Mohamed Othman","year":"2018","venue":"IEEE SYSTEMS JOURNAL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ferdosian et al., 2014] Nasim Ferdosian, Mohamed Oth- man, Borhanuddin Mohd Ali, and Kweh Yeah Lun. Greedy-knapsack algorithm for optimal downlink re- source allocation in lte networks. Wireless Networks, 2014.","title":"Greedy-knapsack algorithm for optimal downlink resource allocation in lte networks","authors":"Nasim Ferdosian, Mohamed Othman, Borhanuddin Mohd Ali, Kweh Yeah Lun","year":"2014","venue":"Wireless Networks","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gandhi et al., 2006] Rajiv Gandhi, Samir Khuller, Srini- vasan Parthasarathy, and Aravind Srinivasan. Dependent rounding and its applications to approximation algorithms. Journal of the ACM, 2006.","title":"Dependent rounding and its applications to approximation algorithms","authors":"Rajiv Gandhi, Samir Khuller, Srinivasan Parthasarathy, Aravind Srinivasan","year":"2006","venue":"Journal of the ACM","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gillen et al., 2018] Stephen Gillen, Christopher Jung, Michael Kearns, and Aaron Roth. Online learning with an unknown fairness metric. In Proceedings of NeurIPS, 2018.","title":"Online learning with an unknown fairness metric","authors":"Stephen Gillen, Christopher Jung, Michael Kearns, Aaron Roth","year":"2018","venue":"Proceedings of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Joseph et al., 2016] Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. Fairness in learning: Classic and contextual bandits. In Proceedings of NIPS, 2016.","title":"Fairness in learning: Classic and contextual bandits","authors":"Matthew Joseph, Michael Kearns, Jamie Morgenstern, Aaron Roth","year":"2016","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2019] Fengjiao Li, Jia Liu, and Bo Ji. Combinato- rial sleeping bandits with fairness constraints. In Proceed- ings of IEEE Infocom, 2019.","title":"Combinatorial sleeping bandits with fairness constraints","authors":"Fengjiao Li, Jia Liu, Bo Ji","year":"2019","venue":"Proceedings of IEEE Infocom","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mahdavi et al., 2012] Mehrdad Mahdavi, Rong Jin, and Tianbao Yang. Trading regret for efficiency: Online con- vex optimization with long term constraints. Journal of Machine Learning Research, 13:2503-2528, 2012.","title":"Trading regret for efficiency: Online convex optimization with long term constraints","authors":"Mehrdad Mahdavi, Rong Jin, Tianbao Yang","year":"2012","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nasim Ferdosian and Ali, 2017] Mohamed Othman Nasim Ferdosian and Borhanuddin Mohd Ali. Downlink scheduling for heterogeneous traffic with gaussian weights in lte-a. In Proceedings of ICC, 2017.","title":"Downlink scheduling for heterogeneous traffic with gaussian weights in lte-a","authors":"Mohamed Othman, Nasim Ferdosian, Borhanuddin Mohd, Ali","year":"2017","venue":"Proceedings of ICC","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[P and B., 2011] Williamson David P and Shmoys David B. The design of approximation algorithms. Cambridge uni- versity press, 2011.","title":"The design of approximation algorithms","authors":"Williamson David, P, Shmoys David, B","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Patil et al., 2019] Vishakha Patil, Ganesh Ghalme, and Vi- neet Nair. Achieving fairness in stochastic multi-armed bandit problem. In arXiv:1905.11260, 2019.","title":"Achieving fairness in stochastic multi-armed bandit problem","authors":"Vishakha Patil, Ganesh Ghalme, Vineet Nair","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sankararaman and Slivkins, 2018] Karthik Abinav Sankararaman and Aleksandrs Slivkins. Combinato- rial semi-bandits with knapsacks. In Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS), 2018.","title":"Combinatorial semi-bandits with knapsacks","authors":"Abinav Karthik, Aleksandrs Sankararaman, Slivkins","year":"2018","venue":"Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Schwartz et al., 2017] Eric M. Schwartz, Eric T. Bradlow, and Peter S. Fader. Customer acquisition via display ad- vertising using multi-armed bandit experiments. Market- ing Science, 2017.","title":"Customer acquisition via display advertising using multi-armed bandit experiments","authors":"Eric M Schwartz, Eric T Bradlow, Peter S Fader","year":"2017","venue":"Marketing Science","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xu et al., 2019] Huanle Xu, Yang Liu, Wing Cheong Lau, Jun Guo, and Alex Liu. Efficient online resource alloca- tion in heterogeneous clusters with machine variability. In Proceedings of IEEE Infocom, 2019.","title":"Efficient online resource allocation in heterogeneous clusters with machine variability","authors":"Huanle Xu, Yang Liu, Cheong Wing, Jun Lau, Alex Guo, Liu","year":"2019","venue":"Proceedings of IEEE Infocom","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu and Neely, 2016] Hao Yu and Michael J. Neely. A low complexity algorithm with O( √ T ) regret and constraint violations for online convex optimization with long term constraints. arXiv preprint:1604.02218, 2016.","title":"A low complexity algorithm with O( √ T ) regret and constraint violations for online convex optimization with long term constraints","authors":"Hao Yu, Michael J Neely","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zheng and Tan, 2014] Liang Zheng and Chee Wei Tan. Op- timal algorithms in wireless utility maximization: Pro- portional fairness decomposition and nonlinear perron- frobenius theory framework. IEEE Transactions on Wire- less Communications, 2014.","title":"Optimal algorithms in wireless utility maximization: Proportional fairness decomposition and nonlinear perronfrobenius theory framework","authors":"Liang Zheng, Chee Wei, Tan","year":"2014","venue":"IEEE Transactions on Wireless Communications","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}