{"type":"ScholarlyArticle","title":"Ad Hoc Teamwork With Behavior Switching Agents","authors":[{"name":"Manish Ravula","slug":"manish-ravula","given":"Manish","family":"Ravula","affiliation":"The University of Texas at Austin , Texas , USA"},{"name":"Shani Alkoby","slug":"shani-alkoby","given":"Shani","family":"Alkoby","affiliation":"The University of Texas at Austin , Texas , USA"},{"name":"Peter Stone","slug":"peter-stone","given":"Peter","family":"Stone","affiliation":"The University of Texas at Austin , Texas , USA"}],"abstract":"As autonomous AI agents proliferate in the real world, they will increasingly need to cooperate with each other to achieve complex goals without always being able to coordinate in advance. This kind of cooperation, in which agents have to learn to cooperate on the fly, is called ad hoc teamwork. Many previous works investigating this setting assumed that teammates behave according to one of many predefined types that is fixed throughout the task. This assumption of stationarity in behaviors, is a strong assumption which cannot be guaranteed in many real-world settings. In this work, we relax this assumption and investigate settings in which teammates can change their types during the course of the task. This adds complexity to the planning problem as now an agent needs to recognize that a change has occurred in addition to figuring out what is the new type of the teammate it is interacting with. In this paper, we present a novel Convolutional-Neural-Network-based Change point Detection (CPD) algorithm for ad hoc teamwork. When evaluating our algorithm on the modified predator prey domain, we find that it outperforms existing Bayesian CPD algorithms.","doi":"10.24963/ijcai.2019/78","url":"https://ijcai.org/proceedings/2019/78","pdf":"https://ijcai.org/proceedings/2019/0078.pdf","markdown":"https://ijcai.org/proceedings/2019/78.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/78","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Agent-based and Multi-agent Systems","track":"Main track","number":78,"pages":{"first":550,"last":556},"keywords":["Agent-based and Multi-agent Systems: Agent Theories and Models","Agent-based and Multi-agent Systems: Coordination and Cooperation","Knowledge Representation and Reasoning: Belief Change","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Albrecht and Ramamoorthy, 2015] Stefano V. Albrecht and Sub- ramanian Ramamoorthy. A Game-Theoretic Model and Best- Response Learning Method for Ad Hoc Coordination in Multia- gent Systems. arXiv, Jun 2015.","title":"Subramanian Ramamoorthy. A Game-Theoretic Model and Best-Response Learning Method for Ad Hoc Coordination in Multiagent Systems","authors":"Stefano V Albrecht","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Albrecht and Stone, 2017] Stefano V Albrecht and Peter Stone. Reasoning about hypothetical agent behaviours and their param- eters. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, pages 547-555. International Foundation for Autonomous Agents and Multiagent Systems, 2017.","title":"Reasoning about hypothetical agent behaviours and their parameters","authors":"V Stefano, Peter Albrecht, Stone","year":"2017","venue":"Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Albrecht and Stone, 2018] Stefano V. Albrecht and Peter Stone. Autonomous agents modelling other agents: A comprehensive survey and open problems. Artificial Intelligence, 258:66-95, 5 2018.","title":"Autonomous agents modelling other agents: A comprehensive survey and open problems","authors":"Stefano V Albrecht, Peter Stone","year":"2018","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Albrecht et al., 2016] Stefano V. Albrecht, Jacob W. Crandall, and Subramanian Ramamoorthy. Belief and truth in hypothesised be- haviours. Artificial Intelligence, 235:63-94, 6 2016.","title":"Belief and truth in hypothesised behaviours","authors":"Stefano V Albrecht, Jacob W Crandall, Subramanian Ramamoorthy","year":"2016","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Aminikhanghahi and Cook, 2017] Samaneh Aminikhanghahi and Diane J Cook. A survey of methods for time series change point detection. Knowledge and information systems, 51(2):339-367, 2017.","title":"Samaneh Aminikhanghahi and Diane J Cook. A survey of methods for time series change point detection","authors":"Cook Aminikhanghahi","year":"2017","venue":"Knowledge and information systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Auer et al., 2002] Peter Auer, Nicol Cesa-Bianchi, and Paul Fis- cher. Finite-time Analysis of the Multiarmed Bandit Problem. Machine Learning, 47(2/3):235-256, 2002.","title":"Finite-time Analysis of the Multiarmed Bandit Problem","authors":"Auer","year":"2002","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Barrett et al., 2011] Samuel Barrett, Peter Stone, and Sarit Kraus. Empirical evaluation of ad hoc teamwork in the pursuit do- main. Autonomous Agents and Multiagent Systems (AAMAS), (May):567-574, 2011.","title":"Empirical evaluation of ad hoc teamwork in the pursuit domain","authors":"Barrett","year":"2011","venue":"Autonomous Agents and Multiagent Systems (AAMAS)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Barrett et al., 2012] Samuel Barrett, Peter Stone, Sarit Kraus, and Avi Rosenfeld. Learning teammate models for ad hoc teamwork. In AAMAS Adaptive Learning Agents (ALA) Workshop, pages 57-63, 2012.","title":"Learning teammate models for ad hoc teamwork","authors":"Barrett","year":"2012","venue":"AAMAS Adaptive Learning Agents (ALA) Workshop","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Benda et al., 1986] Benda, V Jagannathan, and R Dodhiawala. On optimal cooperation of knowledge sources -an empirical investi- gation. (BCSG201028), 1986.","title":"On optimal cooperation of knowledge sources -an empirical investigation","authors":"V Benda, Jagannathan, Dodhiawala","year":"1986","venue":"BCSG201028","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Brown, 1951] George W Brown. Iterative solution of games by fictitious play. Activity analysis of production and allocation, 13(1):374-376, 1951.","title":"Iterative solution of games by fictitious play","authors":"George W Brown","year":"1951","venue":"Activity analysis of production and allocation","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fearnhead and Liu, 2007] Paul Fearnhead and Zhen Liu. On- line inference for multiple changepoint problems. Journal of the Royal Statistical Society. Series B: Statistical Methodology, 69(4):589-605, 2007.","title":"Online inference for multiple changepoint problems","authors":"Liu ; ] Fearnhead, Fearnhead Paul, Zhen Liu","year":"2007","venue":"Journal of the Royal Statistical Society. Series B: Statistical Methodology","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Forney, 1973] G. D. Forney. The viterbi algorithm. Proc. IEEE, 61(3):268-278, Mar 1973.","title":"The viterbi algorithm","authors":"G D Forney","year":"1973","venue":"Proc. IEEE","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Fridman, 2018] Lex Fridman. 6.S094: Deep Learning for Self- Driving Cars. (January), 2018.","title":"Lex Fridman. 6.S094: Deep Learning for Self-Driving Cars","authors":"Fridman","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hausknecht et al., 2016] Matthew Hausknecht, Prannoy Muppa- raju, Sandeep Subramanian, Shivaram Kalyanakrishnan, and Pe- ter Stone. Half field offense: An environment for multiagent learning and ad hoc teamwork. In AAMAS Adaptive Learning Agents (ALA) Workshop, May 2016.","title":"Half field offense: An environment for multiagent learning and ad hoc teamwork","authors":"Matthew Hausknecht, Prannoy Mupparaju, Sandeep Subramanian, Shivaram Kalyanakrishnan, Peter Stone","year":"2016","venue":"AAMAS Adaptive Learning Agents (ALA) Workshop","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hernandez-Leal et al., 2017] Pablo Hernandez-Leal, Yusen Zhan, Matthew E Taylor, L Enrique Sucar, and Enrique de Cote. Effi- ciently detecting switches against non-stationary opponents. Au- tonomous Agents and Multi-Agent Systems, 31(4):767-789, 7 2017.","title":"Efficiently detecting switches against non-stationary opponents","authors":"Hernandez-Leal","year":"2017","venue":"Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kuncheva, 2013] Ludmila I. Kuncheva. Change Detection in Streaming Multivariate Data Using Likelihood Detectors. IEEE Transactions on Knowledge and Data Engineering, 25(5):1175- 1180, 5 2013.","title":"Change Detection in Streaming Multivariate Data Using Likelihood Detectors","authors":"I Ludmila, Kuncheva","year":null,"venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436, 2015.","title":"Deep learning","authors":"Yann Lecun, Yoshua Bengio, Geoffrey Hinton","year":"2015","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[MacAlpine et al., 2014] Patrick MacAlpine, Katie Genter, Samuel Barrett, and Peter Stone. The RoboCup 2013 drop-in player chal- lenges: Experiments in ad hoc teamwork. In 2014 IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems, pages 382-387. IEEE, 9 2014.","title":"The RoboCup 2013 drop-in player challenges: Experiments in ad hoc teamwork","authors":"Patrick Macalpine, Katie Genter, Samuel Barrett, Peter Stone","year":"2014","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Meng et al., 2017] Zhao Meng, Lili Mou, and Zhi Jin. Hierarchi- cal RNN with Static Sentence-Level Attention for Text-Based Speaker Change Detection. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management -CIKM '17, pages 2203-2206, New York, New York, USA, 2017. ACM Press.","title":"Hierarchical RNN with Static Sentence-Level Attention for Text-Based Speaker Change Detection","authors":"Zhao Meng, Lili Mou, Zhi Jin","year":"2017","venue":"Proceedings of the 2017 ACM on Conference on Information and Knowledge Management -CIKM '17","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Niekum et al., 2015] Scott Niekum, Sarah Osentoski, Christo- pher G. Atkeson, and Andrew G. Barto. Online Bayesian change- point detection for articulated motion models. Proceedings - IEEE International Conference on Robotics and Automation, 2015-June(June):1468-1475, 2015.","title":"Online Bayesian changepoint detection for articulated motion models","authors":"Scott Niekum, Sarah Osentoski, Christopher G Atkeson, Andrew G Barto","year":"2015","venue":"Proceedings -IEEE International Conference on Robotics and Automation","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nikolaidis and Shah, 2013] Stefanos Nikolaidis and Julie Shah. Human-robot cross-training: Computational formulation, mod- eling and evaluation of a human team training strategy. 2013 8th ACM/IEEE International Conference on Human-Robot Interac- tion (HRI), pages 33-40, Mar 2013.","title":"Human-robot cross-training: Computational formulation, modeling and evaluation of a human team training strategy","authors":"Stefanos Nikolaidis, Julie Shah","year":"2013","venue":"8th ACM/IEEE International Conference on Human-Robot Interaction (HRI)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Nikolaidis et al., 2014] Stefanos Nikolaidis, Keren Gu, Ramya Ramakrishnan, and Julie Shah. Efficient Model Learning for Human-Robot Collaborative Tasks. arXiv, May 2014.","title":"Efficient Model Learning for Human-Robot Collaborative Tasks","authors":"Stefanos Nikolaidis, Keren Gu, Ramya Ramakrishnan, Julie Shah","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"E. S. Page. Continuous Inspection Schemes. Biometrika, 41(1-2):100-115, 6 1954.","title":"Continuous Inspection Schemes","authors":"E S Page","year":"1954","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Reeves et al., 2007] Jaxk Reeves, Jien Chen, Xiaolan L. Wang, Robert Lund, and Qi Qi Lu. A review and comparison of change- point detection techniques for climate data. Journal of Applied Meteorology and Climatology, 46(6):900-915, 2007.","title":"A review and comparison of changepoint detection techniques for climate data","authors":"Reeves","year":"2007","venue":"Journal of Applied Meteorology and Climatology","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Stone et al., 2010] Peter Stone, Gal A. Kaminka, Sarit Kraus, and Jeffrey S. Rosenschein. Ad Hoc Autonomous Agent Teams: Col- laboration without Pre-Coordination. Twenty-Fourth AAAI Con- ference on Artificial Intelligence, 7 2010.","title":"Ad Hoc Autonomous Agent Teams: Collaboration without Pre-Coordination","authors":"Peter Stone, A Gal, Sarit Kaminka, Jeffrey S Kraus, Rosenschein","year":"2010","venue":"Twenty-Fourth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sugiyama et al., 2008] Masashi Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul V. Buenau, and Motoaki Kawanabe. Di- rect Importance Estimation with Model Selection and Its Appli- cation to Covariate Shift Adaptation, 2008.","title":"Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation","authors":"Masashi Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul V Buenau, Motoaki Kawanabe","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tambe, 1996] Milind Tambe. Tracking dynamic team activity. In Proceedings of the Thirteenth National Conference on Artificial Intelligence -Volume 1, AAAI'96, pages 80-87. AAAI Press, 1996.","title":"Tracking dynamic team activity","authors":"Milind Tambe","year":"1996","venue":"Proceedings of the Thirteenth National Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xuan, 2007] Xiang Xuan. Bayesian inference on change point problems. PhD thesis, University of British Columbia, 2007.","title":"Bayesian inference on change point problems","authors":"Xiang Xuan","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}