{"type":"ScholarlyArticle","title":"Multi-modal Predicate Identification using Dynamically Learned Robot Controllers","authors":[{"name":"Saeid Amiri","slug":"saeid-amiri","given":"Saeid","family":"Amiri","affiliation":"Cleveland State University , Cleveland , OH 44115 USA"},{"name":"Suhua Wei","slug":"suhua-wei","given":"Suhua","family":"Wei","affiliation":"Cleveland State University , Cleveland , OH 44115 USA"},{"name":"Shiqi Zhang","slug":"shiqi-zhang","given":"Shiqi","family":"Zhang","affiliation":"Cleveland State University , Cleveland , OH 44115 USA"},{"name":"Jivko Sinapov","slug":"jivko-sinapov","given":"Jivko","family":"Sinapov","affiliation":"Tufts University , Medford , MA 02155 USA"},{"name":"Jesse Thomason","slug":"jesse-thomason","given":"Jesse","family":"Thomason","affiliation":"The University of Texas at Austin , Austin , TX 78712 USA"},{"name":"Peter Stone","slug":"peter-stone","given":"Peter","family":"Stone","affiliation":"The University of Texas at Austin , Austin , TX 78712 USA"}],"abstract":"Intelligent robots frequently need to explore the objects in their working environments. Modern sensors have enabled robots to learn object properties via perception of multiple modalities. However, object exploration in the real world poses a challenging trade-off between information gains and exploration action costs. Mixed observability Markov decision process (MOMDP) is a framework for planning under uncertainty, while accounting for both fully and partially observable components of the state. Robot perception frequently has to face such mixed observability. This work enables a robot equipped with an arm to dynamically construct query-oriented MOMDPs for multi-modal predicate identification (MPI) of objects. The robot's behavioral policy is learned from two datasets collected using real robots. Our approach enables a robot to explore object properties in a way that is significantly faster while improving accuracies in comparison to existing methods that rely on hand-coded exploration strategies.","doi":"10.24963/ijcai.2018/645","url":"https://ijcai.org/proceedings/2018/645","pdf":"https://ijcai.org/proceedings/2018/0645.pdf","markdown":"https://ijcai.org/proceedings/2018/645.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/645","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Planning and Scheduling","track":"Main track","number":645,"pages":{"first":4638,"last":4645},"keywords":["Robotics: Learning in Robotics","Planning and Scheduling: Robot Planning","Robotics: Developmental Robotics","Planning and Scheduling: POMDPs"],"videos":[],"references":[{"position":1,"raw":"[Alomari et al., 2017] Muhannad Alomari, Paul Duckworth, David C. Hogg, and Anthony G. Cohn. Natural language acquisition and grounding for embodied robotic systems. In AAAI, February 2017.","title":"Natural language acquisition and grounding for embodied robotic systems","authors":"Muhannad Alomari, Paul Duckworth, David C Hogg, Anthony G Cohn","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2016] Min Chen, Emilio Frazzoli, David Hsu, and Wee Sun Lee. Pomdp-lite for robust robot planning under uncertainty. In ICRA, pages 5427-5433, 2016.","title":"Pomdp-lite for robust robot planning under uncertainty","authors":"Chen","year":"2016","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chu et al., 2015] Vivian Chu, Ian McMahon, Lorenzo Ri- ano, Craig G McDonald, et al. Robotic learning of haptic adjectives through physical interaction. Robotics and Au- tonomous Systems, 63:279-292, 2015.","title":"Robotic learning of haptic adjectives through physical interaction","authors":"Vivian Chu, Ian Mcmahon, Lorenzo Riano, Craig G Mcdonald","year":"2015","venue":"Robotics and Autonomous Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Denil et al., 2017] Misha Denil, Pulkit Agrawal, Tejas D Kulkarni, Tom Erez, Peter Battaglia, and Nando de Fre- itas. Learning to perform physics experiments via deep reinforcement learning. In International Conference on Learning Representations, 2017.","title":"Learning to perform physics experiments via deep reinforcement learning","authors":"Misha Denil, Pulkit Agrawal, D Tejas, Tom Kulkarni, Peter Erez, Nando Battaglia, De Freitas","year":"2017","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Eidenberger and Scharinger, 2010] Robert Eidenberger and Josef Scharinger. Active perception and scene modeling by planning with probabilistic 6d object poses. In IROS, pages 1036-1043. IEEE, 2010.","title":"Robert Eidenberger and Josef Scharinger. Active perception and scene modeling by planning with probabilistic 6d object poses","authors":"Scharinger Eidenberger","year":"2010","venue":"IROS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fishel and Loeb, 2012] Jeremy Fishel and Gerald Loeb. Bayesian exploration for intelligent identification of tex- tures. Frontiers in Neurorobotics, 6:4, 2012.","title":"Bayesian exploration for intelligent identification of textures","authors":"Loeb Fishel, Jeremy Fishel, Gerald Loeb","year":"2012","venue":"Frontiers in Neurorobotics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gao et al., 2016] Yang Gao, Lisa Anne Hendricks, Kather- ine J Kuchenbecker, and Trevor Darrell. Deep learning for tactile understanding from visual and haptic data. In ICRA, pages 536-543. IEEE, 2016.","title":"Deep learning for tactile understanding from visual and haptic data","authors":"Yang Gao, Lisa Anne Hendricks, Katherine J Kuchenbecker, Trevor Darrell","year":"2016","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Högman et al., 2013] Virgile Högman, Marten Björkman, and Danica Kragic. Interactive object classification using sensorimotor contingencies. In IROS. IEEE, 2013.","title":"Interactive object classification using sensorimotor contingencies","authors":"Marten Virgile Högman, Danica Björkman, Kragic","year":"2013","venue":"IROS. IEEE","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kaelbling et al., 1998] Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intel- ligence, 101(1):99-134, 1998.","title":"Planning and acting in partially observable stochastic domains","authors":"Leslie Pack, Kaelbling Michael L Littman, Anthony R Cassandra","year":"1998","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Khandelwal et al., 2017] Piyush Khandelwal, Shiqi Zhang, Jivko Sinapov, Matteo Leonetti, Jesse Thomason, et al. Bwibots: A platform for bridging the gap between ai and human-robot interaction research. The International Jour- nal of Robotics Research, 36(5-7):635-659, 2017.","title":"Bwibots: A platform for bridging the gap between ai and human-robot interaction research","authors":"Piyush Khandelwal, Shiqi Zhang, Jivko Sinapov, Matteo Leonetti, Jesse Thomason","year":"2017","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Krishnamurthy and Kollar, 2013] Jayant Krishnamurthy and Thomas Kollar. Jointly learning to parse and perceive: Connecting natural language to the physical world. Trans- actions of the Association for Computational Linguistics, 1:193-206, 2013.","title":"Jointly learning to parse and perceive: Connecting natural language to the physical world","authors":"Jayant Krishnamurthy, Thomas Kollar","year":"2013","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kurniawati et al., 2009] Hanna Kurniawati, David Hsu, and Wee Sun Lee. SARSOP: efficient point-based POMDP planning by approximating optimally reachable belief spaces. In Robotics: Science and Systems Conference, pages 65-72. The MIT Press, 2009.","title":"SARSOP: efficient point-based POMDP planning by approximating optimally reachable belief spaces","authors":"Hanna Kurniawati, David Hsu, Wee Sun, Lee","year":"2009","venue":"Robotics: Science and Systems Conference","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Matuszek et al., 2012] Cynthia Matuszek, Nicholas FitzGerald, Luke Zettlemoyer, Liefeng Bo, and Dieter Fox. A joint model of language and perception for grounded attribute learning. In ICML, 2012.","title":"A joint model of language and perception for grounded attribute learning","authors":"Cynthia Matuszek, Nicholas Fitzgerald, Luke Zettlemoyer, Liefeng Bo, Dieter Fox","year":"2012","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ong et al., 2010] Sylvie CW Ong, Shao Wei Png, David Hsu, and Wee Sun Lee. Planning under uncertainty for robotic tasks with mixed observability. The International Journal of Robotics Research, 29(8):1053-1068, 2010.","title":"Planning under uncertainty for robotic tasks with mixed observability","authors":"C W Sylvie, Shao Ong, David Wei Png, Wee Hsu, Lee Sun","year":"2010","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pajarinen and Kyrki, 2015] Joni Pajarinen and Ville Kyrki. Robotic manipulation of multiple objects as a POMDP. Artificial Intelligence, 2015.","title":"Joni Pajarinen and Ville Kyrki. Robotic manipulation of multiple objects as a POMDP","authors":"Kyrki Pajarinen","year":"2015","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Puterman, 1994] Martin L Puterman. Markov decision pro- cesses: discrete stochastic dynamic programming. John Wiley & Sons, 1994.","title":"Markov decision processes: discrete stochastic dynamic programming","authors":"Martin L Puterman","year":"1994","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rebguns et al., 2011] Antons Rebguns, Daniel Ford, and Ian R Fasel. Infomax control for acoustic exploration of objects by a mobile robot. In Lifelong Learning, 2011.","title":"Infomax control for acoustic exploration of objects by a mobile robot","authors":"Antons Rebguns, Daniel Ford, Ian R Fasel","year":"2011","venue":"Lifelong Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sinapov and Stoytchev, 2009] Jivko Sinapov and Alexander Stoytchev. From acoustic object recognition to object cat- egorization by a humanoid robot. In the RSS Workshop: Mobile Manipulation in Human Environments, 2009.","title":"Jivko Sinapov and Alexander Stoytchev. From acoustic object recognition to object categorization by a humanoid robot","authors":"Stoytchev Sinapov","year":"2009","venue":"the RSS Workshop: Mobile Manipulation in Human Environments","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sinapov et al., 2014a] Jivko Sinapov, Connor Schenck, Ker- rick Staley, Vladimir Sukhoy, and Alexander Stoytchev. Grounding semantic categories in behavioral interactions: Experiments with 100 objects. Robotics and Autonomous Systems, 62(5):632-645, 2014.","title":"Grounding semantic categories in behavioral interactions: Experiments with 100 objects","authors":"Sinapov","year":"2014","venue":"Robotics and Autonomous Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sinapov et al., 2014b] Jivko Sinapov, Connor Schenck, and Alexander Stoytchev. Learning relational object categories using behavioral exploration and multimodal perception. In ICRA, pages 5691-5698, 2014.","title":"Learning relational object categories using behavioral exploration and multimodal perception","authors":"Sinapov","year":"2014","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sinapov et al., 2016] Jivko Sinapov, Priyanka Khante, Maxwell Svetlik, and Peter Stone. Learning to order objects using haptic and proprioceptive exploratory behaviors. In IJCAI, 2016.","title":"Learning to order objects using haptic and proprioceptive exploratory behaviors","authors":"Jivko Sinapov, Priyanka Khante, Maxwell Svetlik, Peter Stone","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/489"},{"position":23,"raw":"[Sridharan et al., 2010] Mohan Sridharan, Jeremy Wyatt, and Richard Dearden. Planning to see: A hierarchi- cal approach to planning visual actions on a robot using POMDPs. Artificial Intelligence, 174(11):704-725, 2010.","title":"Planning to see: A hierarchical approach to planning visual actions on a robot using POMDPs","authors":"Mohan Sridharan, Jeremy Wyatt, Richard Dearden","year":"2010","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Thomason et al., 2016] Jesse Thomason, Jivko Sinapov, Maxwell Svetlik, Peter Stone, and Raymond J Mooney. Learning multi-modal grounded linguistic semantics by playing I Spy. In IJCAI, 2016.","title":"Learning multi-modal grounded linguistic semantics by playing I Spy","authors":"Jesse Thomason, Jivko Sinapov, Maxwell Svetlik, Peter Stone, Raymond J Mooney","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/491"},{"position":25,"raw":"[Thomason et al., 2018] Jesse Thomason, Jivko Sinapov, Raymond Mooney, and Peter Stone. Guiding exploratory behaviors for multi-modal grounding of linguistic descrip- tions. In AAAI, 2018.","title":"Guiding exploratory behaviors for multi-modal grounding of linguistic descriptions","authors":"Thomason","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Whitney et al., 2016] David Whitney, Miles Eldon, John Oberlin, and Stefanie Tellex. Interpreting Multimodal Re- ferring Expressions in Real Time. In ICRA, 2016.","title":"Interpreting Multimodal Referring Expressions in Real Time","authors":"David Whitney, Miles Eldon, John Oberlin, Stefanie Tellex","year":"2016","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang and Stone, 2015] Shiqi Zhang and Peter Stone. CORPP: Commonsense Reasoning and Probabilistic Plan- ning, as Applied to Dialog with a Mobile Robot. In AAAI, 2015.","title":"CORPP: Commonsense Reasoning and Probabilistic Planning, as Applied to Dialog with a Mobile Robot","authors":"Stone Zhang, Shiqi Zhang, Peter Stone","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang and Stone, 2017] Shiqi Zhang and Peter Stone. In- tegrated commonsense reasoning and probabilistic plan- ning. In Proceedings of the 5th Workshop on Planning and Robotics (PlanRob), pages 111-114, 2017.","title":"Integrated commonsense reasoning and probabilistic planning","authors":"Stone Zhang, Shiqi Zhang, Peter Stone","year":"2017","venue":"Proceedings of the 5th Workshop on Planning and Robotics (PlanRob)","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2017] Shiqi Zhang, Piyush Khandelwal, and Peter Stone. Dynamically constructed (PO)MDPs for adaptive robot planning. In AAAI, 2017.","title":"Dynamically constructed (PO)MDPs for adaptive robot planning","authors":"Shiqi Zhang, Piyush Khandelwal, Peter Stone","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}