{"type":"ScholarlyArticle","title":"Sequential Vaccine Allocation with Delayed Feedback","authors":[{"name":"Yichen Xiao","slug":"yichen-xiao","given":"Yichen","family":"Xiao","affiliation":"University of Warwick"},{"name":"Han-Ching Ou","slug":"han-ching-ou","given":"Han-Ching","family":"Ou","affiliation":"Harvard University"},{"name":"Haipeng Chen","slug":"haipeng-chen","given":"Haipeng","family":"Chen","affiliation":"Harvard University"},{"name":"Van Thieu Nguyen","slug":"van-thieu-nguyen","given":"Van Thieu","family":"Nguyen","affiliation":"FSOFT QAI"},{"name":"Long Tran-Thanh","slug":"long-tran-thanh","given":"Long","family":"Tran-Thanh","affiliation":"University of Warwick; FSOFT QAI"}],"abstract":"In this work we consider the problem of how to best allocate a limited supply of vaccines in the aftermath of an infectious disease outbreak by viewing the problem as a sequential game between a learner and an environment (specifically, a bandit problem). The difficulty of this problem lies in the fact that the payoff of vaccination cannot be directly observed, making it difficult to compare the relative effectiveness of vaccination on different population groups. Currently used vaccination policies make recommendations based on mathematical modelling and ethical considerations. These policies are static, and do not adapt as conditions change. Our aim is to design and evaluate an algorithm which can make use of routine surveillance data to dynamically adjust its recommendation. We evaluate the performance of our approach by applying it to a simulated epidemic of a disease based on real-world COVID-19 data, and show that our vaccination policy was able to perform better than existing vaccine allocation policies.\r\nIn particular, we show that with our allocation method, we can reduce the number of required vaccination by at least 50% in order to keep the peak number of hospitalised patients below a certain threshold. Also, when the same batch sizes are used, our method can reduce the peak number of hospitalisation by up to 20%.\r\nWe also demonstrate that our vaccine allocation does not vary the number of batches per group much, making it socially more acceptable (as it reduces uncertainty, hence results in better and more interpretable communication).","doi":"10.24963/ijcai.2022/722","url":"https://ijcai.org/proceedings/2022/722","pdf":"https://ijcai.org/proceedings/2022/0722.pdf","markdown":"https://ijcai.org/proceedings/2022/722.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/722","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Research Papers","track":"AI for Good","number":722,"pages":{"first":5199,"last":5205},"keywords":["Machine Learning: Reinforcement Learning","Uncertainty in AI: Sequential Decision Making","AI Ethics, Trust, Fairness: Societal Impact of AI","Machine Learning: Online Learning","Multidisciplinary Topics and Applications: Health and Medicine"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38985273","length_sec":484,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bistritz et al., 2019] Ilai Bistritz, Zhengyuan Zhou, Xi Chen, Nicholas Bambos, and Jose Blanchet. Exp3 learning in adversarial bandits with delayed feedback. Advances in neural information processing systems, 2019.","title":"Exp3 learning in adversarial bandits with delayed feedback","authors":"Ilai Bistritz, Zhengyuan Zhou, Xi Chen, Nicholas Bambos, Jose Blanchet","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Laura S. P. Bloomfield. Global mapping of landscape fragmentation, human-animal interactions, and livelihood behaviors to prevent the next pandemic. Agric Human Values, pages 1-2, May 2020.","title":"Global mapping of landscape fragmentation, human-animal interactions, and livelihood behaviors to prevent the next pandemic","authors":"S P Laura, Bloomfield","year":"2020","venue":"Agric Human Values","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Carpetta et al., 2021] Stefano Carpetta, Mark Pearson, Francesca Colombo, Ruth Lopert, Guillaume Dedet, and Martin Wenzl. Access to covid-19 vaccines: Global ap- proaches in a global crisis, Mar 2021.","title":"Access to covid-19 vaccines: Global approaches in a global crisis","authors":"Stefano Carpetta, Mark Pearson, Francesca Colombo, Ruth Lopert, Guillaume Dedet, Martin Wenzl","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cesa-Bianchi et al., 2016] Nicolò Cesa-Bianchi, Claudio Gentile, Yishay Mansour, and Alberto Minora. Delay and cooperation in nonstochastic bandits. In Conference on Learning Theory, pages 605-622. PMLR, 2016.","title":"Delay and cooperation in nonstochastic bandits","authors":"Nicolò Cesa-Bianchi, Claudio Gentile, Yishay Mansour, Alberto Minora","year":"2016","venue":"Conference on Learning Theory","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cesa-Bianchi et al., 2018] Nicolò Cesa-Bianchi, Claudio Gentile, and Yishay Mansour. Nonstochastic bandits with composite anonymous feedback. In Sébastien Bubeck, Vianney Perchet, and Philippe Rigollet, editors, Proceed- ings of the 31st Conference On Learning Theory, vol- ume 75 of Proceedings of Machine Learning Research, pages 750-773. PMLR, 06-09 Jul 2018.","title":"Nonstochastic bandits with composite anonymous feedback","authors":"Cesa-Bianchi","year":"2018","venue":"Proceedings of the 31st Conference On Learning Theory","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Davies et al., 2020] Nicholas G Davies, Petra Klepac, Yang Liu, Kiesha Prem, Mark Jit, and Rosalind M Eggo. Age- dependent effects in the transmission and control of covid- 19 epidemics. Nature medicine, 26(8):1205-1211, 2020.","title":"Agedependent effects in the transmission and control of covid-19 epidemics","authors":"Petra Nicholas G Davies, Yang Klepac, Kiesha Liu, Mark Prem, Rosalind M Jit, Eggo","year":"2020","venue":"Nature medicine","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ehreth, 2003] J. Ehreth. The global value of vaccination. Vaccine, 21(7-8):596-600, Jan 2003.","title":"The global value of vaccination","authors":"J Ehreth","year":"2003","venue":"Vaccine","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Foy et al., 2021] Brody H. Foy, Brian Wahl, Kayur Mehta, Anita Shet, Gautam I. Menon, and Carl Britto. Comparing covid-19 vaccine allocation strategies in india: A mathe- matical modelling study. International Journal of Infec- tious Diseases, 103:431-438, 2021.","title":"Comparing covid-19 vaccine allocation strategies in india: A mathematical modelling study","authors":"H Brody, Brian Foy, Kayur Wahl, Anita Mehta, Shet, I Gautam, Carl Menon, Britto","year":"2021","venue":"International Journal of Infectious Diseases","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gergely Neu et al., 2010] András György Gergely Neu, Csaba Szepesvári, and András Antos. Online markov de- cision processes under bandit feedback. In Proceedings of the Twenty-Fourth Annual Conference on Neural Informa- tion Processing Systems, 2010.","title":"Online markov decision processes under bandit feedback","authors":"Gergely Neu","year":"2010","venue":"Proceedings of the Twenty-Fourth Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Frank Houghton. Geography, global pan- demics & air travel: faster, fuller, further & more frequent. Journal of infection and public health, 12(3):448, 2019.","title":"Geography, global pandemics & air travel: faster, fuller, further & more frequent","authors":"Frank Houghton","year":"2019","venue":"Journal of infection and public health","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Joulani et al., 2013] Pooria Joulani, Andras Gyorgy, and Csaba Szepesvári. Online learning under delayed feed- back. In International Conference on Machine Learning, pages 1453-1461. PMLR, 2013.","title":"Online learning under delayed feedback","authors":"Joulani","year":"2013","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kaelbling et al., 1998] Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial intel- ligence, 101(1-2):99-134, 1998.","title":"Planning and acting in partially observable stochastic domains","authors":"Leslie Pack, Kaelbling Michael L Littman, Anthony R Cassandra","year":"1998","venue":"Artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lattimore and Szepesvári, 2020] Tor Lattimore and Csaba Szepesvári. Bandit Algorithms. Cambridge University Press, 2020.","title":"Tor Lattimore and Csaba Szepesvári. Bandit Algorithms","authors":"Szepesvári Lattimore","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Matrajt et al., 2020] Laura Matrajt, Julia Eaton, Tiffany Le- ung, and Elizabeth R. Brown. Vaccine optimization for covid-19: Who to vaccinate first? Science Advances, 7(6), 2020.","title":"Vaccine optimization for covid-19: Who to vaccinate first?","authors":"Laura Matrajt, Julia Eaton, Tiffany Leung, Elizabeth R Brown","year":"2020","venue":"Science Advances","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[McMichael et al., 1996] Anthony J McMichael, J. A Haines, Rudolf Slooff, R Sari Kovats, World Health Or- ganization. Office of Global, Integrated Environmental Health, and WHO/WMO/UNEP Task Group on Health Impact Assessement of Climate Change. Climate change and human health : an assessment / prepared by a task group on behalf of the world health organization, the world meteorological association and the united nations environ- ment programme ; editors : A. j. mcmichael ... [et al.], 1996. National Academies of Sciences et al., 2020]","title":"Climate change and human health : an assessment / prepared by a task group on behalf of the world health organization, the world meteorological association and the united nations environment programme","authors":"Anthony J Mcmichael, J A Haines, Rudolf Slooff, Sari Kovats","year":"1996","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"E.M. Na- tional Academies of Sciences, H.M. Division, B.P.H.P.H. Practice, B.H.S. Policy, C.E.A.V.N. Coronavirus, B. Kahn, L. Brown, W. Foege, and H. Gayle. Framework for Eq- uitable Allocation of COVID-19 Vaccine. National Academies Press, 2020.","title":"Framework for Equitable Allocation of COVID-19 Vaccine","authors":"E M Academies Of Sciences, H M Division, B P H P H Practice, B H S Policy, C E A V N Coronavirus, B Kahn, L Brown, W Foege, H Gayle","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Noh et al., 2021] E. B. Noh, H. K. Nam, and H. Lee. Which Group Should be Vaccinated First?: A Systematic Review. Infect Chemother, 53(2):261-270, Jun 2021.","title":"Which Group Should be Vaccinated First?: A Systematic Review","authors":"E B Noh, H K Nam, H Lee","year":"2021","venue":"Infect Chemother","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pritchard et al., 2021] Emma Pritchard, Philippa C Matthews, Nicole Stoesser, David W Eyre, Owen Gethings, Karina-Doris Vihta, Joel Jones, Thomas House, Harper VanSteenHouse, Iain Bell, et al. Impact of vaccination on new sars-cov-2 infections in the united kingdom. Nature medicine, 27(8):1370-1378, 2021.","title":"Impact of vaccination on new sars-cov-2 infections in the united kingdom","authors":"Emma Pritchard, Philippa C Matthews, Nicole Stoesser, David W Eyre, Owen Gethings, Karina-Doris Vihta, Joel Jones, Thomas House, Harper Vansteenhouse, Iain Bell","year":"2021","venue":"Nature medicine","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shim, 2021] Eunha Shim. Optimal allocation of the limited covid-19 vaccine supply in south korea. Journal of Clini- cal Medicine, 10(4), 2021.","title":"Optimal allocation of the limited covid-19 vaccine supply in south korea","authors":"Eunha Shim, Shim","year":"2021","venue":"Journal of Clinical Medicine","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sutton and Barto, 2018] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.","title":"Reinforcement learning: An introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vernade et al., 2017] Claire Vernade, Olivier Cappé, and Vianney Perchet. Stochastic bandit models for delayed conversions. arXiv preprint arXiv:1706.09186, 2017.","title":"Stochastic bandit models for delayed conversions","authors":"Claire Vernade, Olivier Cappé, Vianney Perchet","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vu et al., 2020] Dong Quan Vu, Patrick Loiseau, Alonso Silva, and Long Tran-Thanh. Path planning problems with side observations-when colonels play hide-and- seek. Proceedings of the AAAI Conference on Artificial Intelligence, 34(02):2252-2259, Apr. 2020.","title":"Path planning problems with side observations-when colonels play hide-andseek","authors":"Dong Quan Vu, Patrick Loiseau, Alonso Silva, Long Tran-Thanh","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}