{"type":"ScholarlyArticle","title":"Shared Autonomy Systems with Stochastic Operator Models","authors":[{"name":"Clarissa Costen","slug":"clarissa-costen","given":"Clarissa","family":"Costen","affiliation":"Oxford Robotics Institute , University of Oxford , UK"},{"name":"Marc Rigter","slug":"marc-rigter","given":"Marc","family":"Rigter","affiliation":"Oxford Robotics Institute , University of Oxford , UK"},{"name":"Bruno Lacerda","slug":"bruno-lacerda","given":"Bruno","family":"Lacerda","affiliation":"Oxford Robotics Institute , University of Oxford , UK"},{"name":"Nick Hawes","slug":"nick-hawes","given":"Nick","family":"Hawes","affiliation":"Oxford Robotics Institute , University of Oxford , UK"}],"abstract":"We consider shared autonomy systems where multiple operators (AI and human), can interact with the environment, e.g. by controlling a robot. The decision problem for the shared autonomy system is to select which operator takes control at each timestep, such that a reward specifying the intended system behaviour is maximised. The performance of the human operator is influenced by unobserved factors, such as fatigue or skill level. Therefore, the system must reason over stochastic models of operator performance. We present a framework for stochastic operators in shared autonomy systems (SO-SAS), where we represent operators using rich, partially observable models. We formalise SO-SAS as a mixed-observability Markov decision process, where environment states are fully observable and internal operator states are hidden. We test SO-SAS on a simulated domain and a computer game, empirically showing it results in better performance compared to traditional formulations of shared autonomy systems.","doi":"10.24963/ijcai.2022/640","url":"https://ijcai.org/proceedings/2022/640","pdf":"https://ijcai.org/proceedings/2022/0640.pdf","markdown":"https://ijcai.org/proceedings/2022/640.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/640","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Planning and Scheduling","track":"Main Track","number":640,"pages":{"first":4614,"last":4620},"keywords":["Humans and AI: Human-AI Collaboration","Agent-based and Multi-agent Systems: Human-Agent Interaction","Planning and Scheduling: Markov Decisions Processes","Planning and Scheduling: Planning under Uncertainty","Robotics: Human Robot Interaction"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987098","length_sec":91,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985241","length_sec":657,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Basich et al., 2020] Connor Basich, Justin Svegliato, Kyle Hollins Wray, Stefan Witwicki, Joydeep Biswas, and Shlomo Zilber- stein. Learning to Optimize Autonomy in Competence-Aware Systems. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '20, pages 123-131, Richland, SC, May 2020. International Founda- tion for Autonomous Agents and Multiagent Systems.","title":"Learning to Optimize Autonomy in Competence-Aware Systems","authors":"Connor Basich, Justin Svegliato, Kyle Hollins Wray, Stefan Witwicki, Joydeep Biswas, Shlomo Zilberstein","year":"2020","venue":"Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '20","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Borovička et al., 2014] Tomáš Borovička, Radim Špetlík, and Karel Rymeš. DataLab Birds -Angry Birds AI, August 2014.","title":"Tomáš Borovička, Radim Špetlík, and Karel Rymeš","authors":"Borovička","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Charles et al., 2018] Jack-Antoine Charles, Caroline P. C. Chanel, Corentin Chauffaut, Pascal Chauvin, and Nicolas Drougard. Human-Agent Interaction Model Learning based on Crowd- sourcing. In Proceedings of the 6th International Conference on Human-Agent Interaction, HAI '18, pages 20-28, New York, NY, USA, December 2018. Association for Computing Machinery.","title":"Human-Agent Interaction Model Learning based on Crowdsourcing","authors":"Jack-Antoine Charles, Caroline P C Chanel, Corentin Chauffaut, Pascal Chauvin, Nicolas Drougard","year":"2018","venue":"Proceedings of the 6th International Conference on Human-Agent Interaction, HAI '18","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cubuktepe et al., 2021] Murat Cubuktepe, Nils Jansen, Mo- hammed Alshiekh, and Ufuk Topcu. Synthesis of Provably Cor- rect Autonomy Protocols for Shared Control. IEEE Transactions on Automatic Control, 66(7):3251-3258, July 2021. Conference Name: IEEE Transactions on Automatic Control.","title":"Synthesis of Provably Correct Autonomy Protocols for Shared Control","authors":"Nils Murat Cubuktepe, Mohammed Jansen, Ufuk Alshiekh, Topcu","year":"2021","venue":"IEEE Transactions on Automatic Control","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dawson and Reid, 1997] Drew Dawson and Kathryn Reid. Fatigue, alcohol and performance impairment. Nature, 388(6639):235-235, July 1997. Bandiera abtest: a Cg type: Na- ture Research Journals Number: 6639 Primary atype: Research Publisher: Nature Publishing Group. [Duchetto et al., 2018]","title":"Fatigue, alcohol and performance impairment","authors":"Dawson, Reid ;, Drew Dawson, Kathryn Reid","year":"1997","venue":"Bandiera abtest: a Cg type: Nature Research Journals Number: 6639 Primary atype","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"Francesco Duchetto, Ayse Kucukyilmaz, Luca Iocchi, and Marc Hanheide. Do Not Make the Same Mis- takes Again and Again: Learning Local Recovery Policies for Navigation From Human Demonstrations. IEEE Robotics and Automation Letters, PP:1-1, July 2018.","title":"Do Not Make the Same Mistakes Again and Again: Learning Local Recovery Policies for Navigation From Human Demonstrations","authors":"Francesco Duchetto, Ayse Kucukyilmaz, Luca Iocchi, Marc Hanheide","year":"2018","venue":"IEEE Robotics and Automation Letters","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Feng et al., 2016] L. Feng, C. Wiltsche, L. Humphrey, and U. Topcu. Synthesis of Human-in-the-Loop Control Protocols for Autonomous Systems. IEEE Transactions on Automation Sci- ence and Engineering, 13(2):450-462, April 2016. Conference Name: IEEE Transactions on Automation Science and Engineer- ing.","title":"Synthesis of Human-in-the-Loop Control Protocols for Autonomous Systems","authors":"L Feng, C Wiltsche, L Humphrey, U Topcu","year":"2016","venue":"IEEE Transactions on Automation Science and Engineering","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fern et al., 2014] A. Fern, S. Natarajan, K. Judah, and P. Tadepalli. A Decision-Theoretic Model of Assistance. Journal of Artificial Intelligence Research, 50:71-104, May 2014.","title":"A Decision-Theoretic Model of Assistance","authors":"A Fern, S Natarajan, K Judah, P Tadepalli","year":"2014","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Javdani et al., 2015] Shervin Javdani, Siddhartha S. Srinivasa, and J. Andrew Bagnell. Shared Autonomy via Hindsight Optimiza- tion. arXiv:1503.07619 [cs], April 2015. arXiv: 1503.07619.","title":"Shared Autonomy via Hindsight Optimization","authors":"Shervin Javdani, Siddhartha S Srinivasa, J Andrew Bagnell","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jean-Baptiste et al., 2015] Emilie M. D. Jean-Baptiste, Pia Rot- shtein, and Martin Russell. POMDP Based Action Planning and Human Error Detection. In Richard Chbeir, Yannis Manolopou- los, Ilias Maglogiannis, and Reda Alhajj, editors, Artificial In- telligence Applications and Innovations, IFIP Advances in Infor- mation and Communication Technology, pages 250-265, Cham, 2015. Springer International Publishing.","title":"POMDP Based Action Planning and Human Error Detection","authors":"M D Emilie, Pia Jean-Baptiste, Martin Rotshtein, Russell","year":"2015","venue":"Artificial Intelligence Applications and Innovations, IFIP Advances in Information and Communication Technology","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jochen Renz, 2021] Jochen Renz. AI Birds.org -Angry Birds AI Competition, 2021.","title":"AI Birds.org -Angry Birds AI Competition","authors":"Jochen Renz","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Junges et al., 2018] Sebastian Junges, Nils Jansen, Joost-Pieter Katoen, Ufuk Topcu, Ruohan Zhang, and Mary Hayhoe. Model Checking for Safe Navigation Among Humans. In Annabelle McIver and Andras Horvath, editors, Quantitative Evaluation of Systems, Lecture Notes in Computer Science, pages 207-222, Cham, 2018. Springer International Publishing.","title":"Model Checking for Safe Navigation Among Humans","authors":"Sebastian Junges, Nils Jansen, Joost-Pieter Katoen, Ufuk Topcu, Ruohan Zhang, Mary Hayhoe","year":"2018","venue":"Quantitative Evaluation of Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ong et al., 2010] Sylvie C. W. Ong, Shao Wei Png, David Hsu, and Wee Sun Lee. Planning under Uncertainty for Robotic Tasks with Mixed Observability. The International Journal of Robotics Research, 29(8):1053-1068, July 2010. Publisher: SAGE Publi- cations Ltd STM. Rigter et al., 2020]","title":"Planning under Uncertainty for Robotic Tasks with Mixed Observability","authors":"C W Sylvie, Shao Ong, David Wei Png, Wee Hsu, Lee Sun","year":"2010","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"Marc Rigter, Bruno Lacerda, and Nick Hawes. A Framework for Learning From Demonstration With Mini- mal Human Effort. IEEE Robotics and Automation Letters, 5(2):2023-2030, April 2020.","title":"A Framework for Learning From Demonstration With Minimal Human Effort","authors":"Marc Rigter, Bruno Lacerda, Nick Hawes","year":"2020","venue":"IEEE Robotics and Automation Letters","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Rosenthal and Veloso, 2012] Stephanie Rosenthal and Manuela Veloso. Mobile robot planning to seek help with spatially- situated tasks. In Proceedings of the Twenty-Sixth AAAI Con- ference on Artificial Intelligence, AAAI'12, pages 2067-2073, Toronto, Ontario, Canada, July 2012. AAAI Press.","title":"Mobile robot planning to seek help with spatiallysituated tasks","authors":"Stephanie Rosenthal, Manuela Veloso","year":"2012","venue":"Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, AAAI'12","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Silver and Veness, 2010] David Silver and Joel Veness. Monte- Carlo planning in large POMDPs. In Proceedings of the 23rd International Conference on Neural Information Processing Sys- tems -Volume 2, NIPS'10, pages 2164-2172, Red Hook, NY, USA, December 2010.","title":"Monte-Carlo planning in large POMDPs","authors":"Veness Silver, David Silver, Joel Veness","year":"2010","venue":"Proceedings of the 23rd International Conference on Neural Information Processing Systems -Volume 2, NIPS'10","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Curran Associates Inc. Wray et al., 2016] Kyle Hollins Wray, Luis Pineda, and Shlomo Zilberstein. Hierarchical approach to transfer of control in semi- autonomous systems. In Proceedings of the Twenty-Fifth Inter- national Joint Conference on Artificial Intelligence, IJCAI'16, pages 517-523, New York, New York, USA, July 2016. AAAI Press.","title":"Hierarchical approach to transfer of control in semiautonomous systems","authors":"Kyle Hollins Wray, Luis Pineda, Shlomo Zilberstein","year":"2016","venue":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI'16","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wray et al., 2017] Kyle Hollins Wray, Stefan J. Witwicki, and Shlomo Zilberstein. Online Decision-Making for Scalable Au- tonomous Systems. In Proceedings of the Twenty-Sixth Interna- tional Joint Conference on Artificial Intelligence, pages 4768- 4774, Melbourne, Australia, August 2017. International Joint Conferences on Artificial Intelligence Organization.","title":"Online Decision-Making for Scalable Autonomous Systems","authors":"Kyle Hollins Wray, Stefan J Witwicki, Shlomo Zilberstein","year":"2017","venue":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/664"},{"position":19,"raw":"[Wu et al., 2017] Bo Wu, Bin Hu, and Hai Lin. Toward efficient manufacturing systems: A trust based human robot collaboration. In 2017 American Control Conference (ACC), pages 1536-1541, May 2017. ISSN: 2378-5861.","title":"Toward efficient manufacturing systems: A trust based human robot collaboration","authors":"Bo Wu, Bin Hu, Hai Lin","year":"2017","venue":"2017 American Control Conference (ACC)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}