{"type":"ScholarlyArticle","title":"Budgeted Multi-Armed Bandits with Multiple Plays","authors":[{"name":"Yingce Xia","slug":"yingce-xia","given":"Yingce","family":"Xia","affiliation":""},{"name":"Tao Qin","slug":"tao-qin","given":"Tao","family":"Qin","affiliation":""},{"name":"Weidong Ma","slug":"weidong-ma","given":"Weidong","family":"Ma","affiliation":""},{"name":"Nenghai Yu","slug":"nenghai-yu","given":"Nenghai","family":"Yu","affiliation":""},{"name":"Tie-Yan Liu","slug":"tie-yan-liu","given":"Tie-Yan","family":"Liu","affiliation":""}],"abstract":"We study the multi-play budgeted multi-armed bandit (MP-BMAB) problem, in which pulling an arm receives both a random reward and a random cost, and a player pulls L( ≥ 1) arms at each round. The player targets at maximizing her total expected reward under a budget constraint B for the pulling costs. We present a multiple ratio confidence bound policy: At each round, we first calculate a truncated upper (lower) confidence bound for the expected reward (cost) of each arm, and then pull the L arms with the maximum ratio of the sum of the upper confidence bounds of rewards to the sum of the lower confidence bounds of costs. We design 0-1 integer linear fractional programming oracle that can pick such the L arms within polynomial time. We prove that the regret of our policy is sublinear in general and is log-linear for certain parameter settings. We further consider two special cases of MP-BMABs: (1) We derive a lower bound for any consistent policy for MP-BMABs with Bernoulli reward and cost distributions. (2) We show that the proposed policy can also solve conventional budgeted MAB problem (a special case of MP-BMABs with L = 1) and provides better theoretical results than existing UCB-based pulling policies.","doi":null,"url":"https://ijcai.org/proceedings/2016/315","pdf":"https://ijcai.org/proceedings/2016/0315.pdf","markdown":"https://ijcai.org/proceedings/2016/315.md","bibtex":"https://ijcai.org/proceedings/2016/bibtex/315","volume":{"slug":"2016","title":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2016","url":"https://ijcai.org/proceedings/2016/","year":2016,"location":"New York, NY, USA","dates":"9–15 July 2016","editors":"Subbarao Kambhampati","isbn":"978-1-57735-770-4; 978-1-57735-771-1","publisher":"AAAI Press / International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":315,"pages":{"first":2210,"last":2215},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}