{"type":"ScholarlyArticle","title":"Adaptive Information Belief Space Planning","authors":[{"name":"Moran Barenboim","slug":"moran-barenboim","given":"Moran","family":"Barenboim","affiliation":"Technion Autonomous Systems Program"},{"name":"Vadim Indelman","slug":"vadim-indelman","given":"Vadim","family":"Indelman","affiliation":"Department of Aerospace Engineering Technion -Israel Institute of Technology , Haifa 32000 , Israel"}],"abstract":"Reasoning about uncertainty is vital in many real-life autonomous systems. However, current state-of-the-art planning algorithms either cannot reason about uncertainty explicitly, or do so with high computational burden. Here, we focus on making informed decisions efficiently, using reward functions that explicitly deal with uncertainty. We formulate an approximation, namely an abstract observation model, that uses an aggregation scheme to alleviate computational costs. We derive bounds on the expected information-theoretic reward function and, as a consequence, on the value function. We then propose a method to refine aggregation to achieve identical action selection in a fraction of the computational time.","doi":"10.24963/ijcai.2022/637","url":"https://ijcai.org/proceedings/2022/637","pdf":"https://ijcai.org/proceedings/2022/0637.pdf","markdown":"https://ijcai.org/proceedings/2022/637.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/637","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Planning and Scheduling","track":"Main Track","number":637,"pages":{"first":4588,"last":4596},"keywords":["Planning and Scheduling: POMDPs","Planning and Scheduling: Planning under Uncertainty","Planning and Scheduling: Robot Planning","Planning and Scheduling: Planning with Incomplete Information","Planning and Scheduling: Planning Algorithms"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986818","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984961","length_sec":241,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Araya et al., 2010] Mauricio Araya, Olivier Buffet, Vincent Thomas, and Franc ¸cois Charpillet. A pomdp extension with belief-dependent rewards. In Advances in Neural In- formation Processing Systems (NIPS), pages 64-72, 2010.","title":"A pomdp extension with belief-dependent rewards","authors":"Mauricio Araya, Olivier Buffet, Vincent Thomas, Franc ¸cois, Charpillet","year":"2010","venue":"Advances in Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boers et al., 2010] Y. Boers, H. Driessen, A. Bagchi, and P. Mandal. Particle filter based entropy. In 2010 13th In- ternational Conference on Information Fusion, pages 1-8, 2010.","title":"Particle filter based entropy","authors":"Y Boers, H Driessen, A Bagchi, P Mandal","year":"2010","venue":"13th International Conference on Information Fusion","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Browne et al., 2012] Cameron B Browne, Edward Pow- ley, Daniel Whitehouse, Simon M Lucas, Peter I Cowl- ing, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in games, 4(1):1-43, 2012.","title":"A survey of monte carlo tree search methods","authors":"Edward Cameron B Browne, Daniel Powley, Whitehouse, M Simon, Peter I Lucas, Philipp Cowling, Stephen Rohlfshagen, Diego Tavener, Spyridon Perez, Simon Samothrakis, Colton","year":"2012","venue":"IEEE Transactions on Computational Intelligence and AI in games","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Burgard et al., 1997] Wolfram Burgard, Dieter Fox, and Se- bastian Thrun. Active mobile robot localization. In Intl. Joint Conf. on AI (IJCAI), pages 1346-1352. Citeseer, 1997.","title":"Active mobile robot localization","authors":"Burgard","year":"1997","venue":"Intl. Joint Conf. on AI (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/1997/194"},{"position":5,"raw":"[Fischer and Tas, 2020] Johannes Fischer and Omer Sahin Tas. Information particle filter tree: An online algorithm for pomdps with belief-based rewards on continuous do- mains. In Intl. Conf. on Machine Learning (ICML), Vi- enna, Austria, 2020. [Flaspohler et al., 2019] Genevieve Flaspohler, Victoria Pre- ston, Anna PM Michel, Yogesh Girdhar, and Nicholas Roy. Information-guided robotic maximum seek-and- sample in partially observable continuous environments. IEEE Robotics and Automation Letters (RA-L), 4(4):3782- 3789, 2019.","title":"Information particle filter tree: An online algorithm for pomdps with belief-based rewards on continuous domains","authors":"Tas Fischer, Johannes Fischer, Omer Sahin Tas, ; Flaspohler","year":"2019","venue":"Intl. Conf. on Machine Learning (ICML), Vienna","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Garg et al., 2019] Neha P Garg, David Hsu, and Wee Sun Lee. Despot-α: Online pomdp planning with large state and observation spaces. In Robotics: Science and Systems (RSS), 2019.","title":"Despot-α: Online pomdp planning with large state and observation spaces","authors":"P Neha, David Garg, Wee Hsu, Lee Sun","year":"2019","venue":"Robotics: Science and Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hoerger and Kurniawati, 2021] Marcus Hoerger and Hanna Kurniawati. An on-line pomdp solver for continuous ob- servation spaces. In IEEE Intl. Conf. on Robotics and Au- tomation (ICRA), pages 7643-7649. IEEE, 2021.","title":"An on-line pomdp solver for continuous observation spaces","authors":"Kurniawati Hoerger, Marcus Hoerger, Hanna Kurniawati","year":"2021","venue":"IEEE Intl. Conf. on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Indelman et al., 2015] V. Indelman, L. Carlone, and F. Del- laert. Planning in the continuous domain: a general- ized belief space approach for autonomous navigation in unknown environments. Intl. J. of Robotics Research, 34(7):849-882, 2015.","title":"Planning in the continuous domain: a generalized belief space approach for autonomous navigation in unknown environments","authors":"V Indelman, L Carlone, F Dellaert","year":"2015","venue":"Intl. J. of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kearns et al., 2002] Michael Kearns, Yishay Mansour, and Andrew Y Ng. A sparse sampling algorithm for near- optimal planning in large markov decision processes. vol- ume 49, pages 193-208. Springer, 2002.","title":"A sparse sampling algorithm for nearoptimal planning in large markov decision processes","authors":"Michael Kearns, Yishay Mansour, Andrew Y Ng","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kim and Eustice, 2014] A. Kim and R. M. Eustice. Active visual SLAM for robotic area coverage: Theory and ex- periment. Intl. J. of Robotics Research, 34(4-5):457-475, 2014.","title":"Active visual SLAM for robotic area coverage: Theory and experiment","authors":"A Kim, R M Eustice","year":"2014","venue":"Intl. J. of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Miller, 2003] Erik G Miller. A new class of entropy esti- mators for multi-dimensional densities. In 2003 IEEE In- ternational Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings.(ICASSP'03)., volume 3, pages III-297. IEEE, 2003.","title":"A new class of entropy estimators for multi-dimensional densities","authors":"Erik G Miller","year":"2003","venue":"IEEE International Conference on Acoustics, Speech, and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Moon et al., 2022] Brady Moon, Satrajit Chatterjee, and Se- bastian Scherer. Tigris: An informed sampling-based al- gorithm for informative path planning, 2022.","title":"Tigris: An informed sampling-based algorithm for informative path planning","authors":"Brady Moon, Satrajit Chatterjee, Sebastian Scherer","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Munos, 2014] Rémi Munos. From Bandits to Monte-Carlo Tree Search: The Optimistic Principle Applied to Opti- mization and Planning. 2014.","title":"From Bandits to Monte-Carlo Tree Search: The Optimistic Principle Applied to Optimization and Planning","authors":"Rémi Munos, Munos","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Papadimitriou and Tsitsiklis, 1987] C. Papadimitriou and J. Tsitsiklis. The complexity of markov decision pro- cesses. Mathematics of operations research, 12(3):441- 450, 1987.","title":"The complexity of markov decision processes","authors":"C Papadimitriou, J Tsitsiklis","year":"1987","venue":"Mathematics of operations research","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Platt et al., 2010] R. Platt, R. Tedrake, L.P. Kaelbling, and T. Lozano-Pérez. Belief space planning assuming maxi- mum likelihood observations. In Robotics: Science and Systems (RSS), pages 587-593, Zaragoza, Spain, 2010.","title":"Belief space planning assuming maximum likelihood observations","authors":"R Platt, R Tedrake, L P Kaelbling, T Lozano-Pérez","year":"2010","venue":"Robotics: Science and Systems (RSS)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Silver and Veness, 2010] David Silver and Joel Veness. Monte-carlo planning in large pomdps. In Advances in Neural Information Processing Systems (NIPS), pages 2164-2172, 2010.","title":"Monte-carlo planning in large pomdps","authors":"Veness Silver, David Silver, Joel Veness","year":"2010","venue":"Advances in Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Somani et al., 2013] Adhiraj Somani, Nan Ye, David Hsu, and Wee Sun Lee. Despot: Online pomdp planning with regularization. In NIPS, volume 13, pages 1772-1780, 2013.","title":"Despot: Online pomdp planning with regularization","authors":"Adhiraj Somani, Nan Ye, David Hsu, Wee Sun, Lee","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Spaan et al., 2015] Matthijs TJ Spaan, Tiago S Veiga, and Pedro U Lima. Decision-theoretic planning under uncer- tainty with information rewards for active cooperative per- ception. Autonomous Agents and Multi-Agent Systems, 29(6):1157-1185, 2015.","title":"Decision-theoretic planning under uncertainty with information rewards for active cooperative perception","authors":"Tiago S Matthijs Tj Spaan, Pedro U Veiga, Lima","year":"2015","venue":"Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sunberg and Kochenderfer, 2018] Zachary Sunberg and Mykel Kochenderfer. Online algorithms for pomdps with continuous state, action, and observation spaces. In Pro- ceedings of the International Conference on Automated Planning and Scheduling, volume 28, 2018.","title":"Online algorithms for pomdps with continuous state, action, and observation spaces","authors":"Kochenderfer Sunberg, Zachary Sunberg, Mykel Kochenderfer","year":"2018","venue":"Proceedings of the International Conference on Automated Planning and Scheduling","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sztyglic and Indelman, 2021] Ori Sztyglic and Vadim In- delman. Online pomdp planning via simplification. arXiv preprint arXiv:2105.05296, 2021.","title":"Ori Sztyglic and Vadim Indelman","authors":"Indelman Sztyglic","year":"2021","venue":"Online pomdp planning via simplification","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sztyglic et al., 2021] O. Sztyglic, A. Zhitnikov, and V. In- delman. Simplified belief-dependent reward mcts planning with guaranteed tree consistency. Technical report, 2021.","title":"Simplified belief-dependent reward mcts planning with guaranteed tree consistency","authors":"O Sztyglic, A Zhitnikov, V Indelman","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Thomas et al., 2021] Vincent Thomas, Geremy Hutin, and Olivier Buffet. Monte carlo information-oriented plan- ning. arXiv preprint arXiv:2103.11345, 2021.","title":"Monte carlo information-oriented planning","authors":"Vincent Thomas, Geremy Hutin, Olivier Buffet","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Thrun et al., 2005] S. Thrun, W. Burgard, and D. Fox. Prob- abilistic Robotics. The MIT press, Cambridge, MA, 2005.","title":"Probabilistic Robotics","authors":"S Thrun, W Burgard, D Fox","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Van Den Berg et al., 2012] J. Van Den Berg, S. Patil, and R. Alterovitz. Motion planning under uncertainty using iterative local optimization in belief space. Intl. J. of Robotics Research, 31(11):1263-1278, 2012.","title":"Motion planning under uncertainty using iterative local optimization in belief space","authors":"Van Den, Berg","year":"2012","venue":"Intl. J. of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Walsh et al., 2010] Thomas Walsh, Sergiu Goschin, and Michael Littman. Integrating sample-based planning and model-based reinforcement learning. In Nat. Conf. on Ar- tificial Intelligence (AAAI), volume 24, 2010.","title":"Integrating sample-based planning and model-based reinforcement learning","authors":"Thomas Walsh, Sergiu Goschin, Michael Littman","year":"2010","venue":"Nat. Conf. on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Ye et al., 2017] Nan Ye, Adhiraj Somani, David Hsu, and Wee Sun Lee. Despot: Online pomdp planning with regu- larization. JAIR, 58:231-266, 2017.","title":"Despot: Online pomdp planning with regularization","authors":"Nan Ye, Adhiraj Somani, David Hsu, Wee Sun, Lee","year":"2017","venue":"JAIR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}