{"type":"ScholarlyArticle","title":"Ranked Prioritization of Groups in Combinatorial Bandit Allocation","authors":[{"name":"Lily Xu","slug":"lily-xu","given":"Lily","family":"Xu","affiliation":"Center for Research on Computation and Society , Harvard University"},{"name":"Arpita Biswas","slug":"arpita-biswas","given":"Arpita","family":"Biswas","affiliation":"Center for Research on Computation and Society , Harvard University"},{"name":"Fei Fang","slug":"fei-fang","given":"Fei","family":"Fang","affiliation":"Institute for Software Research , Carnegie Mellon University"},{"name":"Milind Tambe","slug":"milind-tambe","given":"Milind","family":"Tambe","affiliation":"Center for Research on Computation and Society , Harvard University"}],"abstract":"Preventing poaching through ranger patrols protects endangered wildlife, directly contributing to the UN Sustainable Development Goal 15 of life on land. Combinatorial bandits have been used to allocate limited patrol resources, but existing approaches overlook the fact that each location is home to multiple species in varying proportions, so a patrol benefits each species to differing degrees. When some species are more vulnerable, we ought to offer more protection to these animals; unfortunately, existing combinatorial bandit approaches do not offer a way to prioritize important species. To bridge this gap, (1) We propose a novel combinatorial bandit objective that trades off between reward maximization and also accounts for prioritization over species, which we call ranked prioritization. We show this objective can be expressed as a weighted linear sum of Lipschitz-continuous reward functions. (2) We provide RankedCUCB, an algorithm to select combinatorial actions that optimize our prioritization-based objective, and prove that it achieves asymptotic no-regret. (3) We demonstrate empirically that RankedCUCB leads to up to 38% improvement in outcomes for endangered species using real-world wildlife conservation data. Along with adapting to other challenges such as preventing illegal logging and overfishing, our no-regret algorithm addresses the general combinatorial bandit problem with a weighted linear objective.","doi":"10.24963/ijcai.2022/723","url":"https://ijcai.org/proceedings/2022/723","pdf":"https://ijcai.org/proceedings/2022/0723.pdf","markdown":"https://ijcai.org/proceedings/2022/723.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/723","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Research Papers","track":"AI for Good","number":723,"pages":{"first":5206,"last":5212},"keywords":["Machine Learning: Reinforcement Learning","Uncertainty in AI: Sequential Decision Making","Planning and Scheduling: Planning under Uncertainty","Multidisciplinary Topics and Applications: Sustainable Development Goals","Machine Learning: Online Learning","Multidisciplinary Topics and Applications: Computational Sustainability"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987129","length_sec":136,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985272","length_sec":542,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Agrawal and Devanur, 2014] Shipra Agrawal and Nikhil R Deva- nur. Bandits with concave rewards and convex knapsacks. In Economics and Computation, 2014.","title":"Bandits with concave rewards and convex knapsacks","authors":"Shipra Agrawal, Nikhil R Devanur","year":"2014","venue":"Economics and Computation","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arponen, 2012] Anni Arponen. Prioritizing species for conserva- tion planning. Biodiversity and Conservation, 21(4), 2012.","title":"Anni Arponen. Prioritizing species for conservation planning","authors":"Arponen","year":"2012","venue":"Biodiversity and Conservation","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Auer et al., 2002] Peter Auer, Nicolo Cesa-Bianchi, and Paul Fis- cher. Finite-time analysis of the multiarmed bandit problem. Ma- chine Learning, 2002.","title":"Finite-time analysis of the multiarmed bandit problem","authors":"Peter Auer, Nicolo Cesa-Bianchi, Paul Fischer","year":"2002","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bastani and Bayati, 2020] Hamsa Bastani and Mohsen Bayati. Online decision making with high-dimensional covariates. OR, 68(1), 2020.","title":"Online decision making with high-dimensional covariates","authors":"Bayati Bastani, Hamsa Bastani, Mohsen Bayati","year":"2020","venue":"OR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cesa-Bianchi and Lugosi, 2012] Nicolo Cesa-Bianchi and Gábor Lugosi. Combinatorial bandits. Journal of Computer and Sys- tem Sciences, 78(5), 2012.","title":"Nicolo Cesa-Bianchi and Gábor Lugosi","authors":"Cesa-Bianchi, Lugosi","year":"2012","venue":"Journal of Computer and System Sciences","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2016] Wei Chen, Yajun Wang, Yang Yuan, and Qinshi Wang. Combinatorial multi-armed bandit and its extension to probabilistically triggered arms. JMLR, 17(1), 2016.","title":"Combinatorial multi-armed bandit and its extension to probabilistically triggered arms","authors":"Wei Chen, Yajun Wang, Yang Yuan, Qinshi Wang","year":"2016","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chohlas-Wood et al., 2021] Alex Chohlas-Wood, Madison Coots, Emma Brunskill, and Sharad Goel. Learning to be fair: A conse- quentialist approach to equitable decision-making. arXiv, 2021.","title":"Learning to be fair: A consequentialist approach to equitable decision-making","authors":"Alex Chohlas-Wood, Madison Coots, Emma Brunskill, Sharad Goel","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Corbett-Davies et al., 2017] Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic decision making and the cost of fairness. In KDD, 2017.","title":"Algorithmic decision making and the cost of fairness","authors":"Corbett-Davies","year":"2017","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dilkina et al., 2017] Bistra Dilkina, Rachel Houtman, et al. Trade- offs and efficiencies in optimal budget-constrained multispecies corridor networks. Conservation Biology, 2017.","title":"Tradeoffs and efficiencies in optimal budget-constrained multispecies corridor networks","authors":"Bistra Dilkina, Rachel Houtman","year":"2017","venue":"Conservation Biology","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dimitrakakis et al., 2017] Christos Dimitrakakis, Yang Liu, David Parkes, and Goran Radanovic. Subjective fairness: Fairness is in the eye of the beholder. Technical report, 2017.","title":"Subjective fairness: Fairness is in the eye of the beholder","authors":"Christos Dimitrakakis, Yang Liu, David Parkes, Goran Radanovic","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dwork et al., 2012] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In ITCS, 2012.","title":"Fairness through awareness","authors":"Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, Richard Zemel","year":"2012","venue":"ITCS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Fang et al., 2016] Fei Fang, Thanh Hong Nguyen, Rob Pickles, Wai Y Lam, Gopalasamy R Clements, Bo An, Amandeep Singh, Milind Tambe, et al. Deploying PAWS: Field optimization of the Protection Assistant for Wildlife Security. In IAAI, 2016.","title":"Deploying PAWS: Field optimization of the Protection Assistant for Wildlife Security","authors":"Fei Fang, Thanh Hong Nguyen, Rob Pickles, Wai Y Lam, Gopalasamy R Clements, Bo An, Amandeep Singh, Milind Tambe","year":"2016","venue":"IAAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gatmiry et al., 2021] Zohreh S Gatmiry, Ashkan Hafezalkotob, Roya Soltani, et al. A security game approach for strategic con- servation against poaching considering food web complexities. Ecological Complexity, 2021.","title":"A security game approach for strategic conservation against poaching considering food web complexities","authors":"S Zohreh, Ashkan Gatmiry, Roya Hafezalkotob, Soltani","year":"2021","venue":"Ecological Complexity","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gilbert, 2008] Natasha Gilbert. A quarter of mammals face extinc- tion. Nature, 455(7214):717-718, 2008.","title":"A quarter of mammals face extinction","authors":"Natasha Gilbert","year":"2008","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Herlihy et al., 2021] Christine Herlihy, Aviva Prins, Aravind Srini- vasan, and John Dickerson. Planning to fairly allocate: Proba- bilistic fairness in the restless bandit setting. arXiv, 2021.","title":"Planning to fairly allocate: Probabilistic fairness in the restless bandit setting","authors":"Christine Herlihy, Aviva Prins, Aravind Srinivasan, John Dickerson","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hossain et al., 2021] Safwan Hossain, Evi Micha, and Nisarg Shah. Fair algorithms for multi-agent multi-armed bandits. In NeurIPS, 2021.","title":"Fair algorithms for multi-agent multi-armed bandits","authors":"Safwan Hossain, Evi Micha, Nisarg Shah","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Joseph et al., 2016] Matthew Joseph, Michael Kearns, Jamie H Morgenstern, and Aaron Roth. Fairness in learning: Classic and contextual bandits. In NeurIPS, 2016.","title":"Fairness in learning: Classic and contextual bandits","authors":"Matthew Joseph, Michael Kearns, Jamie H Morgenstern, Aaron Roth","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Joseph et al., 2018] Matthew Joseph, Michael Kearns, Jamie Mor- genstern, Seth Neel, and Aaron Roth. Meritocratic fairness for infinite and contextual bandits. In AIES, 2018.","title":"Meritocratic fairness for infinite and contextual bandits","authors":"Matthew Joseph, Michael Kearns, Jamie Morgenstern, Seth Neel, Aaron Roth","year":"2018","venue":"AIES","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kearns et al., 2017] Michael Kearns, Aaron Roth, and Zhi- wei Steven Wu. Meritocratic fairness for cross-population se- lection. In ICML, 2017.","title":"Meritocratic fairness for cross-population selection","authors":"Michael Kearns, Aaron Roth, Zhiwei Steven Wu","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kleinberg et al., 2018] Jon Kleinberg, Jens Ludwig, Sendhil Mul- lainathan, and Ashesh Rambachan. Algorithmic fairness. In AEA Papers and Proceedings, 2018.","title":"Algorithmic fairness","authors":"Kleinberg","year":"2018","venue":"AEA Papers and Proceedings","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lattimore and Szepesvári, 2020] Tor Lattimore and Csaba Szepesvári. Bandit algorithms. Cambridge University Press, 2020.","title":"Tor Lattimore and Csaba Szepesvári. Bandit algorithms","authors":"Szepesvári Lattimore","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lattimore et al., 2015] Tor Lattimore, Koby Crammer, and Csaba Szepesvári. Linear multi-resource allocation with semi-bandit feedback. In NeurIPS, 2015.","title":"Linear multi-resource allocation with semi-bandit feedback","authors":"Lattimore","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Li et al., 2019] Fengjiao Li, Jia Liu, and Bo Ji. Combinatorial sleeping bandits with fairness constraints. IEEE Transactions on Network Science and Engineering, 7(3), 2019.","title":"Combinatorial sleeping bandits with fairness constraints","authors":"Fengjiao Li, Jia Liu, Bo Ji","year":"2019","venue":"IEEE Transactions on Network Science and Engineering","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Liu et al., 2017] Yang Liu, Goran Radanovic, Christos Dimi- trakakis, Debmalya Mandal, and David C Parkes. Calibrated fair- ness in bandits. FAT-ML Workshop, 2017.","title":"Calibrated fairness in bandits","authors":"Yang Liu, Goran Radanovic, Christos Dimitrakakis, Debmalya Mandal, David C Parkes","year":"2017","venue":"FAT-ML Workshop","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Misra et al., 2019] Kanishka Misra, Eric M Schwartz, and Jacob Abernethy. Dynamic online pricing with incomplete information using multiarmed bandit experiments. Marketing Science, 2019.","title":"Dynamic online pricing with incomplete information using multiarmed bandit experiments","authors":"Kanishka Misra, Eric M Schwartz, Jacob Abernethy","year":"2019","venue":"Marketing Science","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Plumptre et al., 2014] Andrew J Plumptre, Richard A Fuller, Ag- grey Rwetsiba, Fredrick Wanyama, Deo Kujirakwinja, Margaret Driciru, et al. Efficiently targeting resources to deter illegal ac- tivities in protected areas. J. of Appl. Ecology, 51(3), 2014.","title":"Efficiently targeting resources to deter illegal activities in protected areas","authors":"Richard A Andrew J Plumptre, Aggrey Fuller, Fredrick Rwetsiba, Deo Wanyama, Margaret Kujirakwinja, Driciru","year":"2014","venue":"J. of Appl. Ecology","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Regan et al., 2008] Helen Regan, Lauren Hierl, et al. Species pri- oritization for monitoring and management in regional multiple species conservation plans. Diversity and Distributions, 2008.","title":"Species prioritization for monitoring and management in regional multiple species conservation plans","authors":"Helen Regan, Lauren Hierl","year":"2008","venue":"Diversity and Distributions","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Rodrigues et al., 2006] Ana SL Rodrigues, John D Pilgrim, John F Lamoreux, Michael Hoffmann, et al. The value of the IUCN Red List for conservation. Trends Ecol. Evol., 21(2), 2006.","title":"The value of the IUCN Red List for conservation","authors":"Ana Sl Rodrigues, John D Pilgrim, John F Lamoreux, Michael Hoffmann","year":"2006","venue":"Trends Ecol. Evol","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Schumann et al., 2022] Candice Schumann, Zhi Lang, Nicholas Mattei, and John P Dickerson. Group fairness in bandit arm se- lection. AAMAS, 2022.","title":"Group fairness in bandit arm selection","authors":"Candice Schumann, Zhi Lang, Nicholas Mattei, John P Dickerson","year":"2022","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Segal et al., 2018] Avi Segal, Yossi Ben David, Joseph Jay Williams, Kobi Gal, and Yaar Shalom. Combining difficulty ranking with multi-armed bandits to sequence educational con- tent. In AIED, 2018.","title":"Combining difficulty ranking with multi-armed bandits to sequence educational content","authors":"Avi Segal, Yossi, Ben David, Joseph Jay Williams, Kobi Gal, Yaar Shalom","year":"2018","venue":"AIED","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Slivkins, 2013] Aleksandrs Slivkins. Dynamic ad allocation: Ban- dits with budgets. arXiv, 2013.","title":"Dynamic ad allocation: Bandits with budgets","authors":"Aleksandrs Slivkins","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tran-Thanh et al., 2010] Long Tran-Thanh, Archie Chapman, En- rique Munoz De Cote, Alex Rogers, et al. Epsilon-first policies for budget-limited multi-armed bandits. In AAAI, 2010.","title":"Epsilon-first policies for budget-limited multi-armed bandits","authors":"Tran-Thanh","year":"2010","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Tran-Thanh et al., 2012] Long Tran-Thanh, Archie Chapman, Alex Rogers, and Nicholas Jennings. Knapsack based optimal policies for budget-limited multi-armed bandits. In AAAI, 2012.","title":"Knapsack based optimal policies for budget-limited multi-armed bandits","authors":"Tran-Thanh","year":"2012","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Verma et al., 2019] Arun Verma, Manjesh K. Hanawal, Arun Ra- jkumar, et al. Censored semi-bandits: A framework for resource allocation with censored feedback. In NeurIPS, 2019.","title":"Censored semi-bandits: A framework for resource allocation with censored feedback","authors":"Arun Verma, Manjesh K Hanawal, Arun Rajkumar","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2021] Lequn Wang, Yiwei Bai, Wen Sun, and Thorsten Joachims. Fairness of exposure in stochastic bandits. In ICML, 2021.","title":"Fairness of exposure in stochastic bandits","authors":"Wang","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Xu et al., 2020] Lily Xu, Shahrzad Gholami, Sara Mc Carthy, Bis- tra Dilkina, Andrew Plumptre, et al. Stay ahead of poachers: Illegal wildlife poaching prediction and patrol planning under un- certainty with field test evaluations. In ICDE, 2020.","title":"Stay ahead of poachers: Illegal wildlife poaching prediction and patrol planning under uncertainty with field test evaluations","authors":"Xu","year":"2020","venue":"ICDE","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Xu et al., 2021a] Lily Xu, Elizabeth Bondi, Fei Fang, Andrew Per- rault, Kai Wang, and Milind Tambe. Dual-mandate patrols: Multi-armed bandits for green security. In AAAI, 2021.","title":"Dual-mandate patrols: Multi-armed bandits for green security","authors":"Xu","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Xu et al., 2021b] Lily Xu, Andrew Perrault, Fei Fang, Haipeng Chen, and Milind Tambe. Robust reinforcement learning under minimax regret for green security. In UAI, 2021.","title":"Robust reinforcement learning under minimax regret for green security","authors":"Xu","year":"2021","venue":"UAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}