{"type":"ScholarlyArticle","title":"I-dual: Solving Constrained SSPs via Heuristic Search in the Dual Space","authors":[{"name":"Felipe Trevizan","slug":"felipe-trevizan","given":"Felipe","family":"Trevizan","affiliation":"CSIRO Australian National University"},{"name":"Sylvie Thiebaux","slug":"sylvie-thiebaux","given":"Sylvie","family":"Thiébaux","affiliation":"CSIRO Australian National University"},{"name":"Pedro Santana","slug":"pedro-santana","given":"Pedro","family":"Santana","affiliation":"MERS Group Massachusetts Institute of Technology"},{"name":"Brian Williams","slug":"brian-williams","given":"Brian","family":"Williams","affiliation":"MERS Group Massachusetts Institute of Technology"}],"abstract":"We consider the problem of generating optimal stochastic policies for Constrained Stochastic Shortest Path problems, which are a natural model for planning under uncertainty for resource-bounded agents with multiple competing objectives. While unconstrained SSPs enjoy a multitude of efficient heuristic search solution methods with the ability to focus on promising areas reachable from the initial state, the state of the art for constrained SSPs revolves around linear and dynamic programming algorithms which explore the entire state space. In this paper, we present i-dual, the first heuristic search algorithm for constrained SSPs. To concisely represent constraints and efficiently decide their violation, i-dual operates in the space of dual variables describing the policy occupation measures. It does so while retaining the ability to use standard value function heuristics computed by well-known methods. Our experiments show that these features enable i-dual to achieve up to two orders of magnitude improvement in run-time and memory over linear programming algorithms.","doi":"10.24963/ijcai.2017/701","url":"https://ijcai.org/proceedings/2017/701","pdf":"https://ijcai.org/proceedings/2017/0701.pdf","markdown":"https://ijcai.org/proceedings/2017/701.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/701","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Best Sister Conferences","track":"Best Sister Conferences","number":701,"pages":{"first":4954,"last":4958},"keywords":["Artificial Intelligence: automated planning","Artificial Intelligence: uncertainty in artificial intelligence"],"videos":[],"references":[{"position":1,"raw":"[Altman, 1999] Eitan Altman. Constrained Markov Deci- sion Processes, volume 7. CRC Press, 1999.","title":"Constrained Markov Decision Processes","authors":"Eitan Altman","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Baier et al., 2004] Christel Baier, Marcus Größer, Martin Leucker, Benedikt Bollig, and Frank Ciesinski. Controller synthesis for probabilistic systems. In Proc. Int. Conf. on Theoretical Computer Science (TCS), 2004.","title":"Controller synthesis for probabilistic systems","authors":"Christel Baier, Marcus Größer, Martin Leucker, Benedikt Bollig, Frank Ciesinski","year":"2004","venue":"Proc. Int. Conf. on Theoretical Computer Science (TCS)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bertsekas and Tsitsiklis, 1991] D.P. Bertsekas and J.N. Tsitsiklis. An Analysis of Stochastic Shortest Path Prob- lems. Mathematics of Operations Research, 16(3):580- 595, 1991.","title":"An Analysis of Stochastic Shortest Path Problems","authors":"D P Bertsekas, J N Tsitsiklis","year":"1991","venue":"Mathematics of Operations Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bertsekas and Tsitsiklis, 1996] D. Bertsekas and J. N. Tsit- siklis. Neuro-Dynamic Programming. Athena Scientific, 1996.","title":"Neuro-Dynamic Programming","authors":"D Bertsekas, J N","year":"1996","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bryce and Buffet, 2008] D. Bryce and O. Buffet. 6th International Planning Competition: Uncertainty Track. In 3rd Int. Probabilistic Planning Competition (IPPC- ICAPS'08), 2008.","title":"6th International Planning Competition: Uncertainty Track","authors":"D Bryce, O Buffet","year":"2008","venue":"3rd Int. Probabilistic Planning Competition (IPPC-ICAPS'08)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[D'Epenoux, 1963] F. D'Epenoux. A probabilistic produc- tion and inventory problem. Management Science, 10:98- 108, 1963.","title":"A probabilistic production and inventory problem","authors":"F D Epenoux","year":"1963","venue":"Management Science","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dolgov and Durfee, 2005] Dmitri A. Dolgov and Ed- mund H. Durfee. Stationary deterministic policies for con- strained mdps with multiple rewards, costs, and discount factors. In Proc. Int. Joint Conf. on Artificial Intelligence, 2005.","title":"Stationary deterministic policies for constrained mdps with multiple rewards, costs, and discount factors","authors":"Dmitri A Dolgov, Edmund H Durfee","year":"2005","venue":"Proc. Int. Joint Conf. on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2005/236"},{"position":8,"raw":"[Kwiatkowska and Parker, 2013] Marta Z. Kwiatkowska and David Parker. Automated verification and strategy synthe- sis for probabilistic systems. In Int. Symp. on Automated Technology for Verification and Analysis, 2013.","title":"Automated verification and strategy synthesis for probabilistic systems","authors":"Marta Z Kwiatkowska, David Parker","year":"2013","venue":"Int. Symp. on Automated Technology for Verification and Analysis","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Mausam and Kolobov, 2012] Mausam and Andrey Kolobov. Planning with Markov Decision Processes. Morgan&Claypool, 2012.","title":"Mausam and Andrey Kolobov. Planning with Markov Decision Processes","authors":"Kolobov Mausam","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Santana et al., 2016] Pedro Santana, Sylvie Thiébaux, and Brian Williams. RAO*: an algorithm for chance con- strained POMDPs. In Proc. AAAI Conference on Artificial Intelligence, 2016.","title":"RAO*: an algorithm for chance constrained POMDPs","authors":"Pedro Santana, Sylvie Thiébaux, Brian Williams","year":"2016","venue":"Proc. AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Sprauel et al., 2014] Jonathan Sprauel, Andrey Kolobov, and Florent Teichteil-Königsbuch. Saturated path- constrained MDP: planning under uncertainty and deter- ministic model-checking constraints. In Proc. AAAI Conf. on Artificial Intelligence, 2014.","title":"Saturated pathconstrained MDP: planning under uncertainty and deterministic model-checking constraints","authors":"Jonathan Sprauel, Andrey Kolobov, Florent Teichteil-Königsbuch","year":"2014","venue":"Proc. AAAI Conf. on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Teichteil-Königsbuch, 2012] Florent Teichteil-Königsbuch. Path-Constrained Markov Decision Processes: bridg- ing the gap between probabilistic model-checking and decision-theoretic planning. In Proc. European Conf. on Artificial Intelligence, 2012.","title":"Path-Constrained Markov Decision Processes: bridging the gap between probabilistic model-checking and decision-theoretic planning","authors":"Florent Teichteil-Königsbuch","year":"2012","venue":"Proc. European Conf. on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Trevizan and Veloso, 2012] F. W. Trevizan and M. M. Veloso. Short-sighted stochastic shortest path problems. In Proc. Int. Conf. on Automated Planning and Schedul- ing, 2012.","title":"Short-sighted stochastic shortest path problems","authors":"F W Trevizan, M M Veloso","year":"2012","venue":"Proc. Int. Conf. on Automated Planning and Scheduling","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Trevizan et al., 2016] F. W. Trevizan, S. Thiébaux, P. San- tana, and B. Williams. Heuristic search in dual space for constrained stochastic shortest path problems. In Proc. Int. Conf. on Automated Planning and Scheduling (ICAPS), 2016.","title":"Heuristic search in dual space for constrained stochastic shortest path problems","authors":"F W Trevizan, S Thiébaux, P Santana, B Williams","year":"2016","venue":"Proc. Int. Conf. on Automated Planning and Scheduling (ICAPS)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Undurti and How, 2010] Aditya Undurti and Jonathan P. How. An online algorithm for constrained pomdps. In Proc. IEEE Int. Conf. on Robotics and Automation, 2010.","title":"An online algorithm for constrained pomdps","authors":"Aditya Undurti, Jonathan P How","year":"2010","venue":"Proc. IEEE Int. Conf. on Robotics and Automation","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}