{"type":"ScholarlyArticle","title":"Robot Task Interruption by Learning to Switch Among Multiple Models","authors":[{"name":"Anahita Mohseni-Kabir","slug":"anahita-mohseni-kabir","given":"Anahita","family":"Mohseni-Kabir","affiliation":"School of Computer Science , Carnegie Mellon University"},{"name":"Manuela Veloso","slug":"manuela-veloso","given":"Manuela","family":"Veloso","affiliation":"School of Computer Science , Carnegie Mellon University"}],"abstract":"While mobile robots reliably perform each service task by accurately localizing and safely navigating avoiding obstacles, they do not respond in any other way to their surroundings. We can make the robots more responsive to their environment by equipping them with models of multiple tasks and a way to interrupt a specific task and switch to another task based on observations. However the challenges of a multiple task model approach include selecting a task model to execute based on observations and having a potentially large set of observations associated with the set of all individual task models. We present a novel two-step solution. First, our approach leverages the tasks' policies and an abstract representation of their states, and learns which task should be executed at each given world state. Secondly, the algorithm uses the learned tasks and identifies the observation stimuli that trigger the interruption of one task and the switch to another task. We show that our solution using the switching stimuli compares favorably to the naive approach of learning a combined model for all the tasks. Moreover, leveraging the stimuli significantly decreases the amount of sensory input processing during the execution of tasks.","doi":"10.24963/ijcai.2018/686","url":"https://ijcai.org/proceedings/2018/686","pdf":"https://ijcai.org/proceedings/2018/0686.pdf","markdown":"https://ijcai.org/proceedings/2018/686.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/686","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Robotics","track":"Main track","number":686,"pages":{"first":4943,"last":4949},"keywords":["Robotics: Learning in Robotics","Machine Learning Applications: Applications of Reinforcement Learning","Robotics: Social Robots","Robotics: Robotics"],"videos":[],"references":[{"position":1,"raw":"[Bacon et al., 2017] Pierre-Luc Bacon, Jean Harb, and Doina Pre- cup. The option-critic architecture. In AAAI, pages 1726-1734, 2017.","title":"Doina Precup. The option-critic architecture","authors":"Bacon","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barto and Mahadevan, 2003] Andrew G. Barto and Sridhar Ma- hadevan. Recent advances in hierarchical reinforcement learning. Discrete Event Dynamic Systems, pages 341-379, 2003.","title":"Recent advances in hierarchical reinforcement learning","authors":"Andrew G Barto, Sridhar Mahadevan","year":"2003","venue":"Discrete Event Dynamic Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bethke et al., 2008] Brett Bethke, Jonathan P. How, and Asuman Ozdaglar. Approximate dynamic programming using support vector regression. In CDC, pages 3811-3816, 2008.","title":"Approximate dynamic programming using support vector regression","authors":"Brett Bethke, Jonathan P How, Asuman Ozdaglar","year":"2008","venue":"CDC","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Choi, 2011] Dongkyu Choi. Reactive goal management in a cog- nitive architecture. Cognitive Systems Research, pages 293-308, 2011.","title":"Reactive goal management in a cognitive architecture","authors":"Dongkyu Choi, Choi","year":"2011","venue":"Cognitive Systems Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Geurts et al., 2006] Pierre Geurts, Damien Ernst, and Louis We- henkel. Extremely randomized trees. Machine learning, pages 3-42, 2006.","title":"Extremely randomized trees","authors":"Geurts","year":"2006","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Humphrys, 1996] Mark Humphrys. Action selection methods us- ing reinforcement learning. From Animals to Animats, pages 135-144, 1996.","title":"Action selection methods using reinforcement learning","authors":"Mark Humphrys","year":"1996","venue":"From Animals to Animats","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jaidee et al., 2012] Ulit Jaidee, Héctor Muñoz-Avila, and David W. Aha. Learning and reusing goal-specific policies for goal-driven autonomy. In ICCBR, pages 182-195, 2012.","title":"Learning and reusing goal-specific policies for goal-driven autonomy","authors":"Ulit Jaidee, Héctor Muñoz-Avila, David W Aha","year":"2012","venue":"ICCBR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Karlsson, 1997] Jonas Karlsson. Learning to solve multiple goals. PhD thesis, University of Rochester, 1997.","title":"Learning to solve multiple goals","authors":"Jonas Karlsson, Karlsson","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kingma and Ba, 2014] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kober et al., 2013] Jens Kober, J. Andrew Bagnell, and Jan Peters. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, pages 1238-1274, 2013.","title":"Reinforcement learning in robotics: A survey","authors":"Jens Kober, J Andrew Bagnell, Jan Peters","year":"2013","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Konidaris, 2016] George Konidaris. Constructing abstraction hier- archies using a skill-symbol loop. In IJCAI, page 1648, 2016.","title":"Constructing abstraction hierarchies using a skill-symbol loop","authors":"George Konidaris","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/236"},{"position":12,"raw":"[Kulkarni et al., 2016] Tejas D. Kulkarni, Karthik Narasimhan, Ar- davan Saeedi, and Josh Tenenbaum. Hierarchical deep reinforce- ment learning: Integrating temporal abstraction and intrinsic mo- tivation. In NIPS, pages 3675-3683, 2016.","title":"Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation","authors":"D Tejas, Karthik Kulkarni, Ardavan Narasimhan, Josh Saeedi, Tenenbaum","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Martinson et al., 2001] Eric Martinson, Alexander Stoytchev, and Ronald C. Arkin. Robot behavioral selection using q-learning. Technical report, Georgia Institute of Technology, 2001.","title":"Robot behavioral selection using q-learning","authors":"Eric Martinson, Alexander Stoytchev, Ronald C Arkin","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Matarić and Michaud, 2008] Maja J. Matarić and Franc ¸ois Michaud. Behavior-based systems. In Springer Handbook of Robotics, pages 891-909. 2008.","title":"Behavior-based systems","authors":"J Maja, Matarić, Franc, Michaud","year":"2008","venue":"Springer Handbook of Robotics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, and David Silver et al. Human-level control through deep reinforce- ment learning. Nature, pages 529-533, 2015.","title":"Human-level control through deep reinforcement learning","authors":"Mnih","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Muñoz-Avila et al., 2015] Héctor Muñoz-Avila, Mark A. Wilson, and David W. Aha. Guiding the ass with goal motivation weights. In Goal Reasoning: Papers from the ACS Workshop, pages 133- 145, 2015.","title":"Guiding the ass with goal motivation weights","authors":"Héctor Muñoz-Avila, Mark A Wilson, David W Aha","year":"2015","venue":"Goal Reasoning: Papers from the ACS Workshop","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nicolescu et al., 2006] Monica Nicolescu, Odest C. Jenkins, and Adam Olenderski. Learning behavior fusion estimation from demonstration. In ROMAN, pages 340-345, 2006.","title":"Learning behavior fusion estimation from demonstration","authors":"Monica Nicolescu, C Odest, Adam Jenkins, Olenderski","year":"2006","venue":"ROMAN","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pedregosa et al., 2011] Fabian Pedregosa, Gaël Varoquaux, and Alexandre Gramfort et al. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, pages 2825- 2830, 2011.","title":"Scikit-learn: Machine learning in Python","authors":"Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pirjanian, 1999] Paolo Pirjanian. Behavior coordination mechanisms-state-of-the-art. Technical report, University of Southern California, 1999.","title":"Behavior coordination mechanisms-state-of-the-art","authors":"Paolo Pirjanian","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Raïevsky and Michaud, 2008] Clément Raïevsky and Franc ¸ois Michaud. Improving situated agents adaptability using interrup- tion theory of emotions. In SAB, pages 301-310, 2008.","title":"Clément Raïevsky and Franc ¸ois Michaud. Improving situated agents adaptability using interruption theory of emotions","authors":"Michaud Raïevsky","year":"2008","venue":"SAB","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sutton and Barto, 1998] Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. Cambridge: MIT press, 1998.","title":"Reinforcement learning: An introduction","authors":"Richard S Sutton, Andrew G Barto","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sutton et al., 1999] Richard S. Sutton, Satinder P. Singh, Doina Precup, and Balaraman Ravindran. Improved switching among temporally abstract actions. In NIPS, pages 1066-1072, 1999.","title":"Improved switching among temporally abstract actions","authors":"Richard S Sutton, P Satinder, Doina Singh, Balaraman Precup, Ravindran","year":"1999","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vattam et al., 2013] Swaroop Vattam, Matthew Klenk, Matthew Molineaux, and David W. Aha. Breadth of approaches to goal reasoning: A research survey. Technical report, Naval Research Lab Washington DC, 2013.","title":"Breadth of approaches to goal reasoning: A research survey","authors":"Swaroop Vattam, Matthew Klenk, Matthew Molineaux, David W Aha","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Veloso et al., 2015] Manuela M. Veloso, Joydeep Biswas, Brian Coltin, and Stephanie Rosenthal. Cobots: Robust symbiotic au- tonomous mobile service robots. In IJCAI, page 4423, 2015.","title":"Cobots: Robust symbiotic autonomous mobile service robots","authors":"Manuela M Veloso, Joydeep Biswas, Brian Coltin, Stephanie Rosenthal","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/656"},{"position":25,"raw":"[Wang et al., 2015] Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Van Hasselt, Marc Lanctot, and Nando De Freitas. Dueling network architectures for deep reinforcement learning. arXiv preprint arXiv:1511.06581, 2015.","title":"Dueling network architectures for deep reinforcement learning","authors":"Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Van Hasselt, Marc Lanctot, Nando De Freitas","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}