{"type":"ScholarlyArticle","title":"Towards a Pretrained Model for Restless Bandits via Multi-arm Generalization","authors":[{"name":"Yunfan Zhao","slug":"yunfan-zhao","given":"Yunfan","family":"Zhao","affiliation":"Harvard University"},{"name":"Nikhil Behari","slug":"nikhil-behari","given":"Nikhil","family":"Behari","affiliation":"Harvard University"},{"name":"Edward Hughes","slug":"edward-hughes","given":"Edward","family":"Hughes","affiliation":"Google"},{"name":"Edwin Zhang","slug":"edwin-zhang","given":"Edwin","family":"Zhang","affiliation":"Harvard University"},{"name":"Dheeraj Nagaraj","slug":"dheeraj-nagaraj","given":"Dheeraj","family":"Nagaraj","affiliation":"Google"},{"name":"Karl Tuyls","slug":"karl-tuyls","given":"Karl","family":"Tuyls","affiliation":"Google"},{"name":"Aparna Taneja","slug":"aparna-taneja","given":"Aparna","family":"Taneja","affiliation":"Google"},{"name":"Milind Tambe","slug":"milind-tambe","given":"Milind","family":"Tambe","affiliation":"Harvard University; Google"}],"abstract":"Restless multi-arm bandits (RMABs) is a class of resource allocation problems with broad application in areas such as healthcare, online advertising, and anti-poaching. We explore several important question such as how to handle arms opting-in and opting-out over time without frequent retraining from scratch, how to deal with continuous state settings with nonlinear reward functions, which appear naturally in practical contexts. We address these questions by developing a pre-trained model (PreFeRMAB) based on a novel combination of three key ideas: (i) to enable fast generalization, we use train agents to learn from each other's experience; (ii) to accommodate streaming RMABs, we derive a new update rule for a crucial $\\lambda$-network; (iii) to handle more complex continuous state settings, we design the algorithm to automatically define an abstract state based on raw observation and reward data. PreFeRMAB allows general zero-shot ability on previously unseen RMABs, and can be fine-tuned on specific instances in a more sample-efficient way than retraining from scratch.  We theoretically prove the benefits of multi-arm generalization and empirically demonstrate the advantages of our approach on several challenging, real-world inspired problems.","doi":"10.24963/ijcai.2024/36","url":"https://ijcai.org/proceedings/2024/36","pdf":"https://ijcai.org/proceedings/2024/0036.pdf","markdown":"https://ijcai.org/proceedings/2024/36.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/36","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":36,"pages":{"first":321,"last":329},"keywords":["Agent-based and Multi-agent Systems: MAS: Multi-agent learning","Agent-based and Multi-agent Systems: MAS: Multi-agent planning","Agent-based and Multi-agent Systems: MAS: Resource allocation","Machine Learning: ML: Few-shot learning"],"videos":[],"references":[{"position":1,"raw":"[Abel et al., 2018] David Abel, Dilip Arumugam, Lucas Lehnert, and Michael Littman. State abstractions for life- long reinforcement learning. In International Conference on Machine Learning, pages 10-19. PMLR, 2018.","title":"State abstractions for lifelong reinforcement learning","authors":"David Abel, Dilip Arumugam, Lucas Lehnert, Michael Littman","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Adelman and Mersereau, 2008] Daniel Adelman and Adam J Mersereau. Relaxations of weakly coupled stochastic dynamic programs. Operations Research, 56(3):712-727, 2008.","title":"Relaxations of weakly coupled stochastic dynamic programs","authors":"Daniel Adelman, Adam J Mersereau","year":"2008","venue":"Operations Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Akbarzadeh and Mahajan, 2019] N Akbarzadeh and A Ma- hajan. Restless bandits with controlled restarts: Indexabil- ity and computation of whittle index., 2019.","title":"Restless bandits with controlled restarts: Indexability and computation of whittle index","authors":"N Akbarzadeh, Mahajan","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Alon et al., 1996] Noga Alon, Yossi Matias, and Mario Szegedy. The space complexity of approximating the fre- quency moments. In Proceedings of the twenty-eighth an- nual ACM symposium on Theory of computing, pages 20- 29, 1996.","title":"The space complexity of approximating the frequency moments","authors":"Yossi Noga Alon, Mario Matias, Szegedy","year":"1996","venue":"Proceedings of the twenty-eighth annual ACM symposium on Theory of computing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Avrachenkov and Borkar, 2022] Konstantin E Avrachenkov and Vivek S Borkar. Whittle index based q-learning for restless bandits with average reward. Automatica, 139:110186, 2022.","title":"Whittle index based q-learning for restless bandits with average reward","authors":"Konstantin E Avrachenkov, S Vivek, Borkar","year":"2022","venue":"Automatica","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Biswas et al., 2021] Arpita Biswas, Gaurav Aggarwal, Pradeep Varakantham, and Milind Tambe. Learn to in- tervene: An adaptive learning policy for restless bandits in application to preventive healthcare. arXiv preprint arXiv:2105.07965, 2021.","title":"Learn to intervene: An adaptive learning policy for restless bandits in application to preventive healthcare","authors":"Arpita Biswas, Gaurav Aggarwal, Pradeep Varakantham, Milind Tambe","year":"2021","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2021/556"},{"position":7,"raw":"[Bommasani et al., 2021] Rishi Bommasani, Drew A Hud- son, Ehsan Adeli, Russ Altman, Simran Arora, Syd- ney von Arx, Michael S Bernstein, Jeannette Bohg, An- toine Bosselut, Emma Brunskill, et al. On the oppor- tunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.","title":"On the opportunities and risks of foundation models","authors":"Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney Von Arx, Jeannette Michael S Bernstein, Antoine Bohg, Emma Bosselut, Brunskill","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chaudhuri and Kalyanakrishnan, 2020] Arghya Roy Chaudhuri and Shivaram Kalyanakrishnan. Regret minimisation in multi-armed bandits using bounded arm memory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 10085-10092, 2020.","title":"Regret minimisation in multi-armed bandits using bounded arm memory","authors":"Arghya Roy, Chaudhuri, Shivaram Kalyanakrishnan","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Devlin et al., 2018] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.04805, 2018. [Dusonchet and Hongler, 2003] Fabrice Dusonchet and M-O Hongler. Continuous-time restless bandit and dynamic scheduling for make-to-stock production. IEEE Transac- tions on Robotics and Automation, 19(6):977-990, 2003.","title":"Continuous-time restless bandit and dynamic scheduling for make-to-stock production","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2003","venue":"IEEE Transactions on Robotics and Automation","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fu et al., 2019] Jing Fu, Yoni Nazarathy, Sarat Moka, and Peter G Taylor. Towards q-learning the whittle index for restless bandits. In 2019 Australian & New Zealand Con- trol Conference (ANZCC), pages 249-254. IEEE, 2019.","title":"Towards q-learning the whittle index for restless bandits","authors":"Jing Fu, Yoni Nazarathy, Sarat Moka, Peter G Taylor","year":"2019","venue":"2019 Australian & New Zealand Control Conference (ANZCC)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gheshlaghi Azar et al., 2013] Mohammad Ghesh- laghi Azar, Rémi Munos, and Hilbert J Kappen. Minimax pac bounds on the sample complexity of reinforcement learning with a generative model. Machine learning, 91:325-349, 2013.","title":"Minimax pac bounds on the sample complexity of reinforcement learning with a generative model","authors":"Mohammad Gheshlaghi Azar, Rémi Munos, Hilbert J Kappen","year":"2013","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hawkins, 2003] Jeffrey Thomas Hawkins. A Langrangian decomposition approach to weakly coupled dynamic opti- mization problems and its applications. PhD thesis, Mas- sachusetts Institute of Technology, 2003.","title":"A Langrangian decomposition approach to weakly coupled dynamic optimization problems and its applications","authors":"Jeffrey Thomas, Hawkins","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hodge and Glazebrook, 2015] David J Hodge and Kevin D Glazebrook. On the asymptotic optimality of greedy index heuristics for multi-action restless bandits. Advances in Applied Probability, 47(3):652-667, 2015.","title":"On the asymptotic optimality of greedy index heuristics for multi-action restless bandits","authors":"J David, Kevin D Hodge, Glazebrook","year":"2015","venue":"Advances in Applied Probability","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jaques et al., 2019] Natasha Jaques, Angeliki Lazari- dou, Edward Hughes, Caglar Gulcehre, Pedro Ortega, DJ Strouse, Joel Z Leibo, and Nando De Freitas. Social influence as intrinsic motivation for multi-agent deep reinforcement learning. In International conference on machine learning, pages 3040-3049. PMLR, 2019.","title":"Social influence as intrinsic motivation for multi-agent deep reinforcement learning","authors":"Natasha Jaques, Angeliki Lazaridou, Edward Hughes, Caglar Gulcehre, Pedro Ortega, Joel Z Strouse, Nando De Leibo, Freitas","year":"2019","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jin et al., 2021] Chi Jin, Qinghua Liu, Yuanhao Wang, and Tiancheng Yu. V-learning-a simple, efficient, decen- tralized algorithm for multiagent rl. arXiv preprint arXiv:2110.14555, 2021.","title":"V-learning-a simple, efficient, decentralized algorithm for multiagent rl","authors":"Jin","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Killian et al., 2021a] Jackson A. Killian, Arpita Biswas, Sanket Shah, and Milind Tambe. Q-learning lagrange poli- cies for multi-action restless bandits. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD '21, page 871-881, New York, NY, USA, 2021. Association for Computing Machinery.","title":"Q-learning lagrange policies for multi-action restless bandits","authors":"Killian","year":"2021","venue":"Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD '21","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Killian et al., 2021b] Jackson A. Killian, Andrew Perrault, and Milind Tambe. Beyond \"to act or not to act\": Fast la- grangian approaches to general multi-action restless ban- dits. In AAMAS, pages 710-718, UK, 2021. AAMAS.","title":"Beyond \"to act or not to act\": Fast lagrangian approaches to general multi-action restless bandits","authors":"Killian","year":"2021","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Killian et al., 2022] Jackson A Killian, Lily Xu, Arpita Biswas, and Milind Tambe. Restless and uncertain: Ro- bust policies for restless bandits via deep multi-agent re- inforcement learning. In Uncertainty in Artificial Intelli- gence, pages 990-1000. PMLR, 2022.","title":"Restless and uncertain: Robust policies for restless bandits via deep multi-agent reinforcement learning","authors":"Lily Jackson A Killian, Arpita Xu, Milind Biswas, Tambe","year":"2022","venue":"Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lefévre, 1981] Claude Lefévre. Optimal control of a birth and death epidemic process. Operations Research, 29(5):971-982, 1981.","title":"Optimal control of a birth and death epidemic process","authors":"Claude Lefévre, Lefévre","year":"1981","venue":"Operations Research","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2006] Lihong Li, Thomas J Walsh, and Michael L Littman. Towards a unified theory of state abstraction for mdps. In AI&M, 2006.","title":"Towards a unified theory of state abstraction for mdps","authors":"Lihong Li, Thomas J Walsh, Michael L Littman","year":"2006","venue":"AI&M","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liau et al., 2018] David Liau, Zhao Song, Eric Price, and Ger Yang. Stochastic multi-armed bandits in constant space. In International Conference on Artificial Intelli- gence and Statistics, pages 386-394. PMLR, 2018.","title":"Stochastic multi-armed bandits in constant space","authors":"David Liau, Zhao Song, Eric Price, Ger Yang","year":"2018","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Littman, 1994] Michael L Littman. Markov games as a framework for multi-agent reinforcement learning. In Ma- chine learning proceedings 1994, pages 157-163. Else- vier, 1994.","title":"Markov games as a framework for multi-agent reinforcement learning","authors":"Michael L Littman","year":"1994","venue":"Machine learning proceedings","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mate et al., 2021] Aditya Mate, Arpita Biswas, Christoph Siebenbrunner, Susobhan Ghosh, and Milind Tambe. Efficient algorithms for finite horizon and streaming restless multi-armed bandit problems. arXiv preprint arXiv:2103.04730, 2021.","title":"Efficient algorithms for finite horizon and streaming restless multi-armed bandit problems","authors":"Aditya Mate, Arpita Biswas, Christoph Siebenbrunner, Susobhan Ghosh, Milind Tambe","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Mate et al., 2022] Aditya Mate, Lovish Madaan, Aparna Taneja, Neha Madhiwalla, Shresth Verma, Gargi Singh, Aparna Hegde, Pradeep Varakantham, and Milind Tambe. Field study in deploying restless multi-armed bandits: As- sisting non-profits in improving maternal and child health. Proceedings of the AAAI Conference on Artificial Intelli- gence, 36:12017-12025, 06 2022.","title":"Field study in deploying restless multi-armed bandits: Assisting non-profits in improving maternal and child health","authors":"Aditya Mate, Lovish Madaan, Aparna Taneja, Neha Madhiwalla, Shresth Verma, Gargi Singh, Aparna Hegde, Pradeep Varakantham, Milind Tambe","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Nakhleh et al., 2021] Khaled Nakhleh, Santosh Ganji, Ping-Chun Hsieh, I Hou, Srinivas Shakkottai, et al. Neur- win: Neural whittle index network for restless bandits via deep rl. Advances in Neural Information Processing Systems, 34:828-839, 2021.","title":"Neurwin: Neural whittle index network for restless bandits via deep rl","authors":"Khaled Nakhleh, Santosh Ganji, Ping-Chun Hsieh, I Hou, Srinivas Shakkottai","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Papadimitriou and Tsitsiklis, 1999] Christos H Papadim- itriou and John N Tsitsiklis. The complexity of optimal queuing network control. Mathematics of Operations Re- search, 24(2):293-305, 1999.","title":"The complexity of optimal queuing network control","authors":"Christos H Papadimitriou, John N Tsitsiklis","year":"1999","venue":"Mathematics of Operations Research","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Ramesh et al., 2021] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning, pages 8821-8831. PMLR, 2021.","title":"Zero-shot text-to-image generation","authors":"Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, Ilya Sutskever","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Riquelme et al., 2018] Carlos Riquelme, George Tucker, and Jasper Snoek. Deep bayesian bandits showdown: An empirical comparison of bayesian deep networks for thompson sampling. In International Conference on Learning Representations, 2018.","title":"Deep bayesian bandits showdown: An empirical comparison of bayesian deep networks for thompson sampling","authors":"Carlos Riquelme, George Tucker, Jasper Snoek","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Shalev-Shwartz and Ben-David, 2014] Shai Shalev- Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms. Cambridge university press, 2014.","title":"Understanding machine learning: From theory to algorithms","authors":"Shai Shalev, -Shwartz, Shai Ben-David","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Shapley, 1953] Lloyd S Shapley. Stochastic games. Proceedings of the national academy of sciences, 39(10):1095-1100, 1953.","title":"Stochastic games","authors":"S Lloyd, Shapley","year":"1953","venue":"Proceedings of the national academy of sciences","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Sinha and Mahajan, 2022] Amit Sinha and Aditya Maha- jan. Robustness of whittle index policy to model approxi- mation. Available at SSRN 4064507, 2022.","title":"Robustness of whittle index policy to model approximation","authors":"Amit Sinha, Aditya Mahajan","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Team et al., 2023] Adaptive Agent Team, Jakob Bauer, Kate Baumli, Satinder Baveja, Feryal Behbahani, Avishkar Bhoopchand, Nathalie Bradley-Schmieg, Michael Chang, Natalie Clay, Adrian Collister, et al. Human-timescale adaptation in an open-ended task space. arXiv preprint arXiv:2301.07608, 2023.","title":"Human-timescale adaptation in an open-ended task space","authors":"Team","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Verma et al., 2023] Shresth Verma, Aditya Mate, Kai Wang, Neha Madhiwalla, Aparna Hegde, Aparna Taneja, and Milind Tambe. Restless multi-armed bandits for maternal and child health: Results from decision-focused learning. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, pages 1312- 1320, 2023.","title":"Restless multi-armed bandits for maternal and child health: Results from decision-focused learning","authors":"Verma","year":"2023","venue":"Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Vershynin, 2018] Roman Vershynin. High-dimensional probability: An introduction with applications in data sci- ence, volume 47. Cambridge university press, 2018.","title":"Roman Vershynin. High-dimensional probability: An introduction with applications in data science","authors":"Vershynin","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2023] Kai Wang, Shresth Verma, Aditya Mate, Sanket Shah, Aparna Taneja, Neha Madhiwalla, Aparna Hegde, and Milind Tambe. Scalable decision-focused learning in restless multi-armed bandits with application to maternal and child health. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 12138-12146, 2023.","title":"Scalable decision-focused learning in restless multi-armed bandits with application to maternal and child health","authors":"Kai Wang, Shresth Verma, Aditya Mate, Sanket Shah, Aparna Taneja, Neha Madhiwalla, Aparna Hegde, Milind Tambe","year":"2023","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Weber and Weiss, 1990] Richard R Weber and Gideon Weiss. On an index policy for restless bandits. Journal of applied probability, 27(3):637-648, 1990.","title":"On an index policy for restless bandits","authors":"Weiss ; Weber, R Richard, Gideon Weber, Weiss","year":"1990","venue":"Journal of applied probability","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Whittle, 1988] Peter Whittle. Restless bandits: Activity al- location in a changing world. Journal of applied probabil- ity, 25(A):287-298, 1988.","title":"Restless bandits: Activity allocation in a changing world","authors":"Peter Whittle, Whittle","year":"1988","venue":"Journal of applied probability","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Xie et al., 2020] Qiaomin Xie, Yudong Chen, Zhaoran Wang, and Zhuoran Yang. Learning zero-sum simultaneous-move markov games using function approx- imation and correlated equilibrium. In Conference on learning theory, pages 3674-3682. PMLR, 2020.","title":"Learning zero-sum simultaneous-move markov games using function approximation and correlated equilibrium","authors":"Qiaomin Xie, Yudong Chen, Zhaoran Wang, Zhuoran Yang","year":"2020","venue":"Conference on learning theory","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Xiong and Li, 2023] Guojun Xiong and Jian Li. Finite- time analysis of whittle index based q-learning for restless multi-armed bandits with neural network function approx- imation. In Thirty-seventh Conference on Neural Informa- tion Processing Systems, 2023.","title":"Finitetime analysis of whittle index based q-learning for restless multi-armed bandits with neural network function approximation","authors":"Guojun Xiong, Jian Li","year":"2023","venue":"Thirty-seventh Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Yaesoubi and Cohen, 2011] Reza Yaesoubi and Ted Co- hen. Generalized markov models of infectious dis- ease spread: A novel framework for developing dynamic health policies. European journal of operational research, 215(3):679-687, 2011.","title":"Reza Yaesoubi and Ted Cohen. Generalized markov models of infectious disease spread: A novel framework for developing dynamic health policies","authors":"Cohen Yaesoubi","year":"2011","venue":"European journal of operational research","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Yu et al., 2020] Wenhao Yu, Jie Tan, Yunfei Bai, Erwin Coumans, and Sehoon Ha. Learning fast adaptation with meta strategy optimization. IEEE Robotics and Automa- tion Letters, 5(2):2950-2957, 2020.","title":"Learning fast adaptation with meta strategy optimization","authors":"Wenhao Yu, Jie Tan, Yunfei Bai, Erwin Coumans, Sehoon Ha","year":"2020","venue":"IEEE Robotics and Automation Letters","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Yu et al., 2022] Chao Yu, Akash Velu, Eugene Vinitsky, Ji- axuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Sys- tems, 35:24611-24624, 2022.","title":"The surprising effectiveness of ppo in cooperative multi-agent games","authors":"Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, Yi Wu","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhang and Frazier, 2021] Xiangyu Zhang and Peter I. Fra- zier. Restless bandits with many arms: Beating the central limit theorem, 2021.","title":"Restless bandits with many arms: Beating the central limit theorem","authors":"Xiangyu Zhang, Peter I Frazier","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhao et al., 2024] Yunfan Zhao, Nikhil Behari, Edward Hughes, Edwin Zhang, Dheeraj Nagaraj, Karl Tuyls, Aparna Taneja, and Milind Tambe. Towards zero shot learning in restless multi-armed bandits. AAMAS (2-page extended abstract), 2024.","title":"Towards zero shot learning in restless multi-armed bandits","authors":"Yunfan Zhao, Nikhil Behari, Edward Hughes, Edwin Zhang, Dheeraj Nagaraj, Karl Tuyls, Aparna Taneja, Milind Tambe","year":"2024","venue":"AAMAS (2-page extended abstract","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}