{"type":"ScholarlyArticle","title":"Limited Resource Allocation in a Non-Markovian World: The Case of Maternal and Child Healthcare","authors":[{"name":"Panayiotis Danassis","slug":"panayiotis-danassis","given":"Panayiotis","family":"Danassis","affiliation":"Harvard University"},{"name":"Shresth Verma","slug":"shresth-verma","given":"Shresth","family":"Verma","affiliation":"Google Research"},{"name":"Jackson A. Killian","slug":"jackson-a-killian","given":"Jackson A.","family":"Killian","affiliation":"Harvard University"},{"name":"Aparna Taneja","slug":"aparna-taneja","given":"Aparna","family":"Taneja","affiliation":"Google Research"},{"name":"Milind Tambe","slug":"milind-tambe","given":"Milind","family":"Tambe","affiliation":"Harvard University; Google Research"}],"abstract":"The success of many healthcare programs depends on participants' adherence. We consider the problem of scheduling interventions in low resource settings (e.g., placing timely support calls from health workers) to increase adherence and/or engagement. Past works have successfully developed several classes of Restless Multi-armed Bandit (RMAB) based solutions for this problem. Nevertheless, all past RMAB approaches assume that the participants' behaviour follows the Markov property. We demonstrate significant deviations from the Markov assumption on real-world data on a maternal health awareness program from our partner NGO, ARMMAN. Moreover, we extend RMABs to continuous state spaces, a previously understudied area. To tackle the generalised non-Markovian RMAB setting we (i) model each participant's trajectory as a time-series, (ii) leverage the power of time-series forecasting models to learn complex patterns and dynamics to predict future states, and (iii) propose the Time-series Arm Ranking Index (TARI) policy, a novel algorithm that selects the RMAB arms that will benefit the most from an intervention, given our future state predictions. We evaluate our approach on both synthetic data, and a secondary analysis on real data from ARMMAN, and demonstrate significant increase in engagement compared to the SOTA, deployed Whittle index solution. This translates to 16.3 hours of additional content listened, 90.8% more engagement drops prevented, and reaching more than twice as many high dropout-risk beneficiaries.","doi":"10.24963/ijcai.2023/660","url":"https://ijcai.org/proceedings/2023/660","pdf":"https://ijcai.org/proceedings/2023/0660.pdf","markdown":"https://ijcai.org/proceedings/2023/660.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/660","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI for Good","track":"AI for Good","number":660,"pages":{"first":5950,"last":5958},"keywords":["AI for Good: Planning and Scheduling","AI for Good: Agent-based and Multi-agent Systems"],"videos":[],"references":[{"position":1,"raw":"[Ariyo et al., 2014] Adebiyi A Ariyo, Adewumi O Adewumi, and Charles K Ayo. Stock price prediction using the arima model. In 2014 UKSim-AMSS 16th international conference on computer modelling and simulation, pages 106-112. IEEE, 2014.","title":"Stock price prediction using the arima model","authors":"Adewumi O Adebiyi A Ariyo, Charles K Adewumi, Ayo","year":"2014","venue":"UKSim-AMSS 16th international conference on computer modelling and simulation","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[ARMMAN, 2023a] ARMMAN. Maternal and newborn health. https://armman.org/maternal-newborn-health/ #1552759270935-9f057600-22c8, 2023. [Online; accessed 8-February-2023].","title":"ARMMAN. Maternal and newborn health","authors":"Armman","year":"2023","venue":null,"doi":null,"url":"https://armman.org/maternal-newborn-health/#1552759270935-9f057600-22c8","internal_url":null},{"position":3,"raw":"[ARMMAN, 2023b] ARMMAN. mmitra. https://armman.org/mmitra/, 2023. [Online; accessed 8- February-2023].","title":"ARMMAN. mmitra","authors":"Armman","year":"2023","venue":null,"doi":null,"url":"https://armman.org/mmitra/,2023","internal_url":null},{"position":4,"raw":"[Auer et al., 2002] Peter Auer, Nicolo Cesa-Bianchi, and Paul Fis- cher. Finite-time analysis of the multiarmed bandit problem. Ma- chine learning, 47:235-256, 2002.","title":"Finite-time analysis of the multiarmed bandit problem","authors":"Peter Auer, Nicolo Cesa-Bianchi, Paul Fischer","year":"2002","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bai et al., 2018] Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and re- current networks for sequence modeling. arXiv preprint arXiv:1803.01271, 2018.","title":"An empirical evaluation of generic convolutional and recurrent networks for sequence modeling","authors":"Bai","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Buchanan, 1991] James M. Buchanan. Opportunity Cost, pages 520-525. Palgrave Macmillan UK, London, 1991.","title":"Opportunity Cost","authors":"M James, Buchanan","year":"1991","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chevillon, 2007] Guillaume Chevillon. Direct multi-step estima- tion and forecasting. Journal of Economic Surveys, 21(4):746- 785, 2007.","title":"Guillaume Chevillon. Direct multi-step estimation and forecasting","authors":"Chevillon","year":"2007","venue":"Journal of Economic Surveys","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chierichetti et al., 2012] Flavio Chierichetti, Ravi Kumar, Prab- hakar Raghavan, and Tamas Sarlos. Are web users really marko- vian? In Proceedings of the 21st International Conference on World Wide Web, WWW '12, page 609-618, New York, NY, USA, 2012. Association for Computing Machinery.","title":"Are web users really markovian?","authors":"Flavio Chierichetti, Ravi Kumar, Prabhakar Raghavan, Tamas Sarlos","year":"2012","venue":"Proceedings of the 21st International Conference on World Wide Web, WWW '12","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Danassis et al., 2023] Panayiotis Danassis, Shresth Verma, Jack- son Killian, Aparna Taneja, and Milind Tambe. Limited re- source allocation in a non-markovian world: The case of mater- nal and child healthcare. arXiv preprint arXiv:2305.12640, 2023. https://arxiv.org/abs/2305.12640.","title":"Limited resource allocation in a non-markovian world: The case of maternal and child healthcare","authors":"Panayiotis Danassis, Shresth Verma, Jackson Killian, Aparna Taneja, Milind Tambe","year":"2023","venue":null,"doi":null,"url":"https://arxiv.org/abs/2305.12640","internal_url":null},{"position":10,"raw":"[Early et al., 2022] Joseph Early, Tom Bewley, Christine Evers, and Sarvapali Ramchurn. Non-markovian reward modelling from tra- jectory labels via interpretable multiple instance learning. arXiv preprint arXiv:2205.15367, 2022.","title":"Non-markovian reward modelling from trajectory labels via interpretable multiple instance learning","authors":"Joseph Early, Tom Bewley, Christine Evers, Sarvapali Ramchurn","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[GatesFoundation, 2020] GatesFoundation. Global progress and projections for maternal mortal- ity: Maternal deaths per 100,000 live births. https://www.gatesfoundation.org/goalkeepers/report/2021- report/progress-indicators/maternal-mortality/, 2020. [Online; accessed 8-February-2023].","title":"GatesFoundation. Global progress and projections for maternal mortality: Maternal deaths per 100,000 live births","authors":null,"year":"2020","venue":null,"doi":null,"url":"https://www.gatesfoundation.org/goalkeepers/report/2021-report/progress-indicators/maternal-mortality/,2020","internal_url":null},{"position":12,"raw":"[Ghosh et al., 2023] Abheek Ghosh, Dheeraj Nagaraj, Manish Jain, and Milind Tambe. Indexability is not enough for whittle: Im- proved, near-optimal algorithms for restless bandits. In Proceed- ings of the 22nd International Conference on Autonomous Agents and Multiagent Systems, AAMAS '23. International Foundation for Autonomous Agents and Multiagent Systems, 2023.","title":"Indexability is not enough for whittle: Improved, near-optimal algorithms for restless bandits","authors":"Abheek Ghosh, Dheeraj Nagaraj, Manish Jain, Milind Tambe","year":"2023","venue":"Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems, AAMAS '23. International Foundation for Autonomous Agents and Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Glazebrook et al., 2006] Kevin D Glazebrook, Diego Ruiz- Hernandez, and Christopher Kirkbride. Some indexable families of restless bandit problems. Advances in Applied Probability, 38(3):643-672, 2006.","title":"Some indexable families of restless bandit problems","authors":"Kevin D Glazebrook, Diego Ruiz-Hernandez, Christopher Kirkbride","year":"2006","venue":"Advances in Applied Probability","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Graves and Schmidhuber, 2005] Alex Graves and Jürgen Schmid- huber. Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural networks, 18(5- 6):602-610, 2005.","title":"Alex Graves and Jürgen Schmidhuber. Framewise phoneme classification with bidirectional lstm and other neural network architectures","authors":"Schmidhuber Graves","year":"2005","venue":"Neural networks","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[HelpMum, 2023] HelpMum. Preventing maternal and infant mor- tality in africa. https://helpmum.org/, 2023. [Online; accessed 8-February-2023].","title":"HelpMum. Preventing maternal and infant mortality in africa","authors":"Helpmum","year":"2023","venue":null,"doi":null,"url":"https://helpmum.org/,2023","internal_url":null},{"position":16,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hsu, 2018] Yu-Pin Hsu. Age of information: Whittle index for scheduling stochastic arrivals. In 2018 IEEE International Sym- posium on Information Theory (ISIT), pages 2634-2638. IEEE, 2018.","title":"Age of information: Whittle index for scheduling stochastic arrivals","authors":"Yu-Pin Hsu","year":"2018","venue":"IEEE International Symposium on Information Theory (ISIT)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kadota et al., 2016] Igor Kadota, Elif Uysal-Biyikoglu, Rahul Singh, and Eytan Modiano. Minimizing the age of information in broadcast wireless networks. In 2016 54th Annual Allerton Con- ference on Communication, Control, and Computing (Allerton), pages 844-851. IEEE, 2016.","title":"Minimizing the age of information in broadcast wireless networks","authors":"Igor Kadota, Elif Uysal-Biyikoglu, Rahul Singh, Eytan Modiano","year":"2016","venue":"54th Annual Allerton Conference on Communication, Control, and Computing (Allerton)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kaur et al., 2020] Jasvir Kaur, Manmeet Kaur, Venkatesan Chakrapani, Jacqui Webster, Joseph Alvin Santos, and Rajesh Kumar. Effectiveness of information technology-enabled 'smart eating'health promotion intervention: A cluster randomized controlled trial. PLoS One, 15(1):e0225892, 2020.","title":"Effectiveness of information technology-enabled 'smart eating'health promotion intervention: A cluster randomized controlled trial","authors":"Jasvir Kaur, Manmeet Kaur, Venkatesan Chakrapani, Jacqui Webster, Joseph, Alvin Santos, Rajesh Kumar","year":"2020","venue":"PLoS One","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kazemi et al., 2019] Seyed Mehran Kazemi, Rishab Goel, Sepehr Eghbali, Janahan Ramanan, Jaspreet Sahota, Sanjay Thakur, Stella Wu, Cathal Smyth, Pascal Poupart, and Marcus Brubaker. Time2vec: Learning a vector representation of time. arXiv preprint arXiv:1907.05321, 2019.","title":"Time2vec: Learning a vector representation of time","authors":"Rishab Seyed Mehran Kazemi, Sepehr Goel, Janahan Eghbali, Jaspreet Ramanan, Sanjay Sahota, Stella Thakur, Cathal Wu, Pascal Smyth, Marcus Poupart, Brubaker","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lai et al., 2018] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal pat- terns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information re- trieval, pages 95-104, 2018.","title":"Modeling long-and short-term temporal patterns with deep neural networks","authors":"Guokun Lai, Wei-Cheng Chang, Yiming Yang, Hanxiao Liu","year":"2018","venue":"The 41st international ACM SIGIR conference on research & development in information retrieval","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2021] Minhao Liu, Ailing Zeng, Zhijian Xu, Qiuxia Lai, and Qiang Xu. Time series is a special sequence: Forecast- ing with sample convolution and interaction. arXiv preprint arXiv:2106.09305, 2021.","title":"Time series is a special sequence: Forecasting with sample convolution and interaction","authors":"Minhao Liu, Ailing Zeng, Zhijian Xu, Qiuxia Lai, Qiang Xu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mate et al., 2022] Aditya Mate, Lovish Madaan, Aparna Taneja, Neha Madhiwalla, Shresth Verma, Gargi Singh, Aparna Hegde, Pradeep Varakantham, and Milind Tambe. Field study in de- ploying restless multi-armed bandits: Assisting non-profits in im- proving maternal and child health. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 12017- 12025, 2022.","title":"Field study in deploying restless multi-armed bandits: Assisting non-profits in improving maternal and child health","authors":"Aditya Mate, Lovish Madaan, Aparna Taneja, Neha Madhiwalla, Shresth Verma, Gargi Singh, Aparna Hegde, Pradeep Varakantham, Milind Tambe","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Matsubara et al., 2014] Yasuko Matsubara, Yasushi Sakurai, Willem G Van Panhuis, and Christos Faloutsos. Funnel: automatic mining of spatially coevolving epidemics. In Pro- ceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 105-114, 2014.","title":"Funnel: automatic mining of spatially coevolving epidemics","authors":"Yasuko Matsubara, Yasushi Sakurai, Willem G Van Panhuis, Christos Faloutsos","year":"2014","venue":"Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Meiss et al., 2010] Mark R. Meiss, Bruno Gonc ¸alves, José J. Ram- asco, Alessandro Flammini, and Filippo Menczer. Agents, book- marks and clicks: A topical model of web navigation. In Proceed- ings of the 21st ACM Conference on Hypertext and Hypermedia, HT '10, page 229-234, New York, NY, USA, 2010. Association for Computing Machinery.","title":"Agents, bookmarks and clicks: A topical model of web navigation","authors":"R Mark, Bruno Meiss, José J Gonc ¸alves, Alessandro Ramasco, Filippo Flammini, Menczer","year":"2010","venue":"Proceedings of the 21st ACM Conference on Hypertext and Hypermedia, HT '10","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Meshram et al., 2016] Rahul Meshram, Aditya Gopalan, and D Manjunath. Optimal recommendation to users that react: On- line learning for a class of pomdps. In 2016 IEEE 55th Confer- ence on Decision and Control (CDC), pages 7210-7215. IEEE, 2016.","title":"Optimal recommendation to users that react: Online learning for a class of pomdps","authors":"Rahul Meshram, Aditya Gopalan, Manjunath","year":"2016","venue":"IEEE 55th Conference on Decision and Control (CDC)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Nishtala et al., 2020] Siddharth Nishtala, Harshavardhan Ka- marthi, Divy Thakkar, Dhyanesh Narayanan, Anirudh Grama, Aparna Hegde, Ramesh Padmanabhan, Neha Madhiwalla, Suresh Chaudhary, Balaraman Ravindran, and Milind Tambe. Missed calls, automated calls and health support: Using ai to im- prove maternal health outcomes by increasing program engage- ment. In AI for Social Good Workshop, 2020.","title":"Missed calls, automated calls and health support: Using ai to improve maternal health outcomes by increasing program engagement","authors":"Siddharth Nishtala, Harshavardhan Kamarthi, Divy Thakkar, Dhyanesh Narayanan, Anirudh Grama, Aparna Hegde, Ramesh Padmanabhan, Neha Madhiwalla, Suresh Chaudhary, Balaraman Ravindran, Milind Tambe","year":"2020","venue":"AI for Social Good Workshop","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Nisthala et al., 2021] Siddhart Nisthala, Lovish Madaan, Aditya Mate, Harshavardhan Kamarthi, Anirudh Grama, Divy Thakkar, Dhyanesh Narayanan, Suresh Chaudhary, Neha Madhiwala, Ramesh Padhmanabhan, Aparna Hegde, Pradeep Varakantham, Balaram Ravindran, and Milind Tambe. Selective intervention planning using restless multi-armed bandits to improve maternal and child health outcomes. In AAMAS workshop on Autonomous Agents for social good, 2021.","title":"Selective intervention planning using restless multi-armed bandits to improve maternal and child health outcomes","authors":"Nisthala","year":"2021","venue":"AAMAS workshop on Autonomous Agents for social good","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Papadimitriou and Tsitsiklis, 1994] Christos H Papadimitriou and John N Tsitsiklis. The complexity of optimal queueing network control. In Proceedings of IEEE 9th Annual Conference on Struc- ture in Complexity Theory, pages 318-322. IEEE, 1994.","title":"The complexity of optimal queueing network control","authors":"Christos H Papadimitriou, John N Tsitsiklis","year":"1994","venue":"Proceedings of IEEE 9th Annual Conference on Structure in Complexity Theory","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Pfammatter et al., 2016] Angela Pfammatter, Bonnie Spring, Nalini Saligram, Raj Davé, Arun Gowda, Linelle Blais, Monika Arora, Harish Ranjani, Om Ganda, Donald Hedeker, et al. mhealth intervention to improve diabetes risk behaviors in india: a prospective, parallel group cohort study. Journal of medical Internet research, 18(8):e207, 2016.","title":"mhealth intervention to improve diabetes risk behaviors in india: a prospective, parallel group cohort study","authors":"Angela Pfammatter, Bonnie Spring, Nalini Saligram, Raj Davé, Arun Gowda, Linelle Blais, Monika Arora, Harish Ranjani, Om Ganda, Donald Hedeker","year":"2016","venue":"Journal of medical Internet research","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Puterman, 2014] Martin L Puterman. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.","title":"Markov decision processes: discrete stochastic dynamic programming","authors":"Martin L Puterman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Qian et al., 2016] Yundi Qian, Chao Zhang, Bhaskar Krishna- machari, and Milind Tambe. Restless poachers: Handling exploration-exploitation tradeoffs in security domains. In Pro- ceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, AAMAS '16, page 123-131, Richland, SC, 2016. International Foundation for Autonomous Agents and Multiagent Systems.","title":"Restless poachers: Handling exploration-exploitation tradeoffs in security domains","authors":"Yundi Qian, Chao Zhang, Bhaskar Krishnamachari, Milind Tambe","year":"2016","venue":"Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, AAMAS '16","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Sinha and Mahajan, 2022] Amit Sinha and Aditya Mahajan. Ro- bustness of whittle index policy to model approximation. Avail- able at SSRN 4064507, 2022.","title":"Robustness of whittle index policy to model approximation","authors":"Amit Sinha, Aditya Mahajan","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Taieb et al., 2012] Souhaib Ben Taieb, Rob J Hyndman, et al. Re- cursive and direct multi-step forecasting: the best of both worlds, volume 19. Department of Econometrics and Business Statistics, Monash Univ., 2012.","title":"Recursive and direct multi-step forecasting: the best of both worlds","authors":"Taieb","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[TheWorldBank, 2023] TheWorldBank. Fact sheet: An adjustment to global poverty lines. https://www.worldbank.org/en/news/factsheet/2022/05/02/fact- sheet-an-adjustment-to-global-poverty-lines, 2023. [Online; accessed 8-February-2023].","title":"TheWorldBank. Fact sheet: An adjustment to global poverty lines","authors":"Theworldbank","year":"2023","venue":null,"doi":null,"url":"https://www.worldbank.org/en/news/factsheet/2022/05/02/fact-sheet-an-adjustment-to-global-poverty-lines,2023","internal_url":null},{"position":36,"raw":"[UN, 2023] UN. Sustainable development goals. https://sdgs.un.org/goals/goal3, 2023. [Online; accessed 8-February-2023].","title":"UN. Sustainable development goals","authors":null,"year":"2023","venue":null,"doi":null,"url":"https://sdgs.un.org/goals/goal3,2023","internal_url":null},{"position":37,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Par- mar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Verma et al., 2023] Shresth Verma, Gargi Singh, Aditya Mate, Paritosh Verma, Sruthi Gorantala, Neha Madhiwalla, Aparna Hegde, Divy Thakkar, Manish Jain, Milind Tambe, and Aparna Taneja. Increasing impact of mobile health programs: Saheli for maternal and childcare. In Innovative Applications of Artificial Intelligence (IAAI), 2023.","title":"Increasing impact of mobile health programs: Saheli for maternal and childcare","authors":"Verma","year":"2023","venue":"Innovative Applications of Artificial Intelligence (IAAI)","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Wang* et al., 2023] Kai Wang*, Shresth Verma*, Aditya Mate, Sanket Shah, Aparna Taneja, Neha Madhiwalla, Aparna Hegde, and Milind Tambe. Scalable decision-focused learning in rest- less multi-armed bandits with application to maternal and child health. In Proceedings of the AAAI Conference on Artificial In- telligence (AAAI), 2023.","title":"Scalable decision-focused learning in restless multi-armed bandits with application to maternal and child health","authors":"Kai Wang, *, Shresth Verma, *, Aditya Mate, Sanket Shah, Aparna Taneja, Neha Madhiwalla, Aparna Hegde, Milind Tambe","year":"2023","venue":"Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Weber and Weiss, 1990] Richard R Weber and Gideon Weiss. On an index policy for restless bandits. Journal of Applied Probabil- ity, 27(3):637-648, 1990.","title":"On an index policy for restless bandits","authors":"Weiss ; Weber, R Richard, Gideon Weber, Weiss","year":"1990","venue":"Journal of Applied Probability","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Wen et al., 2022] Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transformers in time series: A survey. arXiv preprint arXiv:2202.07125, 2022.","title":"Transformers in time series: A survey","authors":"Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, Liang Sun","year":"2022","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2023/759"},{"position":42,"raw":"[Whittle, 1988] Peter Whittle. Restless bandits: Activity allocation in a changing world. Journal of Applied Probability, 25(A):287- 298, 1988.","title":"Restless bandits: Activity allocation in a changing world","authors":"Peter Whittle, Whittle","year":"1988","venue":"Journal of Applied Probability","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wu et al., 2020] Neo Wu, Bradley Green, Xue Ben, and Shawn O'Banion. Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317, 2020.","title":"Deep transformer models for time series forecasting: The influenza prevalence case","authors":"Wu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wu et al., 2021] Haixu Wu, Jiehui Xu, Jianmin Wang, and Ming- sheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems, 34:22419-22430, 2021.","title":"Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting","authors":"Haixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Zeng et al., 2023] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI Conference on Artificial Intelligence, 2023.","title":"Are transformers effective for time series forecasting","authors":"Ailing Zeng, Muxi Chen, Lei Zhang, Qiang Xu","year":"2023","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Zhou et al., 2021] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. In- former: Beyond efficient transformer for long sequence time- series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106-11115, 2021.","title":"Informer: Beyond efficient transformer for long sequence timeseries forecasting","authors":"Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, Wancai Zhang","year":"2021","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}