{"type":"ScholarlyArticle","title":"Beyond Strong-Cyclic: Doing Your Best in Stochastic Environments","authors":[{"name":"Benjamin Aminof","slug":"benjamin-aminof","given":"Benjamin","family":"Aminof","affiliation":"TU Vienna , Austria"},{"name":"Giuseppe De Giacomo","slug":"giuseppe-de-giacomo","given":"Giuseppe","family":"De Giacomo","affiliation":"University of Rome \"La Sapienza\" , Italy"},{"name":"Sasha Rubin","slug":"sasha-rubin","given":"Sasha","family":"Rubin","affiliation":"University of Sydney , Australia"},{"name":"Florian Zuleger","slug":"florian-zuleger","given":"Florian","family":"Zuleger","affiliation":"TU Vienna , Austria"}],"abstract":"``Strong-cyclic policies\" were introduced to formalize trial-and-error strategies and are known to work in Markovian stochastic domains, i.e., they guarantee that the goal is reached with probability 1. We introduce ``best-effort\" policies for (not necessarily Markovian) stochastic domains. These generalize strong-cyclic policies by taking advantage of stochasticity even if the goal cannot be reached with probability 1. We compare such policies with optimal policies, i.e., policies that maximize the probability that the goal is achieved, and show that optimal policies are best-effort, but that the converse is false in general. With this framework at hand, we revisit the foundational problem of what it means to plan in nondeterministic domains when the nondeterminism has a stochastic nature. We show that one can view a nondeterministic planning domain as a representation of infinitely many stochastic domains with the same support but different probabilities, and that for temporally extended goals expressed in LTL/LTLf a finite-state best-effort policy in one of these domains is best-effort in each of the domains. In particular, this gives an approach for finding such policies that reduces to solving finite-state MDPs with LTL/LTLf goals. All this shows that ``best-effort\" policies are robust to changes in the probabilities, as long as the support is unchanged.","doi":"10.24963/ijcai.2022/350","url":"https://ijcai.org/proceedings/2022/350","pdf":"https://ijcai.org/proceedings/2022/0350.pdf","markdown":"https://ijcai.org/proceedings/2022/350.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/350","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Knowledge Representation and Reasoning","track":"Main Track","number":350,"pages":{"first":2525,"last":2531},"keywords":["Planning and Scheduling: Theoretical Foundations of Planning","Knowledge Representation and Reasoning: Reasoning about actions"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987009","length_sec":73,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985152","length_sec":714,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Aminof et al., 2020a] Benjamin Aminof, Giuseppe De Gia- como, Alessio Lomuscio, Aniello Murano, and Sasha Ru- bin. Synthesizing strategies under expected and excep- tional environment behaviors. In IJCAI, 2020.","title":"Synthesizing strategies under expected and exceptional environment behaviors","authors":"Aminof","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/232"},{"position":2,"raw":"[Aminof et al., 2020b] Benjamin Aminof, Giuseppe De Gia- como, and Sasha Rubin. Stochastic fairness and language- theoretic fairness in planning on nondeterministic do- mains. In ICAPS, 2020.","title":"Stochastic fairness and languagetheoretic fairness in planning on nondeterministic domains","authors":"Aminof","year":"2020","venue":"ICAPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Aminof et al., 2021a] Benjamin Aminof, Giuseppe De Gia- como, Alessio Lomuscio, Aniello Murano, and Sasha Ru- bin. Synthesizing best-effort strategies under hierarchical environment specifications. In KR, 2021.","title":"Synthesizing best-effort strategies under hierarchical environment specifications","authors":"Aminof","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Aminof et al., 2021b] Benjamin Aminof, Giuseppe De Gia- como, and Sasha Rubin. Best-effort synthesis: Doing your best is not harder than giving up. In IJCAI, 2021.","title":"Best-effort synthesis: Doing your best is not harder than giving up","authors":"Aminof","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/243"},{"position":5,"raw":"[Bacchus and Kabanza, 2000] Fahiem Bacchus and Frodu- ald Kabanza. Using temporal logics to express search con- trol knowledge for planning. Artif. Intell., 116(1-2), 2000.","title":"Fahiem Bacchus and Froduald Kabanza. Using temporal logics to express search control knowledge for planning","authors":"Kabanza Bacchus","year":"2000","venue":"Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Baier and Kwiatkowska, 1998] Christel Baier and Marta Z. Kwiatkowska. On the verification of qualitative properties of probabilistic processes under fairness constraints. Inf. Process. Lett., 66(2):71-79, 1998.","title":"On the verification of qualitative properties of probabilistic processes under fairness constraints","authors":"Christel Baier, Marta Z Kwiatkowska","year":"1998","venue":"Inf. Process. Lett","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Baier and McIlraith, 2006] Jorge A. Baier and Sheila A. McIlraith. Planning with first-order temporally extended goals using heuristic search. In AAAI, 2006.","title":"Planning with first-order temporally extended goals using heuristic search","authors":"Jorge A Baier, Sheila A Mcilraith","year":"2006","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Baier et al., 2008] Christel Baier, Joost-Pieter Katoen, and Kim Guldstrand Larsen. Principles of Model Checking. The MIT Press, 2008.","title":"Principles of Model Checking","authors":"Christel Baier, Joost-Pieter Katoen, Kim Guldstrand Larsen","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Belle and Levesque, 2016] Vaishak Belle and Hector J. Levesque. Foundations for generalized planning in un- bounded stochastic domains. In KR, 2016.","title":"Vaishak Belle and Hector J. Levesque. Foundations for generalized planning in unbounded stochastic domains","authors":"Belle, Levesque","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Berwanger, 2007] Dietmar Berwanger. Admissibility in in- finite games. In STACS, 2007.","title":"Admissibility in infinite games","authors":"Dietmar Berwanger","year":"2007","venue":"STACS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Bhattacharya and Waymire, 2021] Rabi Bhattacharya and Edward C Waymire. Random Walk, Brownian Motion, and Martingales. Springer, 2021.","title":"Random Walk, Brownian Motion, and Martingales","authors":"Rabi Bhattacharya, Edward C Waymire","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Bianco and de Alfaro, 1995] Andrea Bianco and Luca de Alfaro. Model checking of probabilistic and nondeter- ministic systems. In FSTTCS, 1995.","title":"Model checking of probabilistic and nondeterministic systems","authors":"De Bianco, Andrea Alfaro, Luca Bianco, De Alfaro","year":"1995","venue":"FSTTCS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Bonet et al., 2010] Blai Bonet, Héctor Palacios, and Hector Geffner. Automatic derivation of finite-state machines for behavior control. In AAAI, 2010.","title":"Automatic derivation of finite-state machines for behavior control","authors":"Blai Bonet, Héctor Palacios, Hector Geffner","year":"2010","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Bonet et al., 2017] Blai Bonet, Giuseppe De Giacomo, Hec- tor Geffner, and Sasha Rubin. Generalized planning: Non- deterministic abstractions and trajectory constraints. In IJ- CAI, 2017.","title":"Generalized planning: Nondeterministic abstractions and trajectory constraints","authors":"Blai Bonet, Giuseppe De Giacomo, Hector Geffner, Sasha Rubin","year":"2017","venue":"IJ-CAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Bonet et al., 2020] Blai Bonet, Giuseppe De Giacomo, Hec- tor Geffner, Fabio Patrizi, and Sasha Rubin. High-level programming via generalized planning and LTL synthesis. In KR, 2020.","title":"High-level programming via generalized planning and LTL synthesis","authors":"Blai Bonet, Giuseppe De Giacomo, Hector Geffner, Fabio Patrizi, Sasha Rubin","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Brenguier et al., 2017] Romain Brenguier, Jean-Franc ¸ois Raskin, and Ocan Sankur. Assume-admissible synthesis. Acta Inf., 54(1), 2017.","title":"Assume-admissible synthesis","authors":"Romain Brenguier, Jean-Franc ¸ois Raskin, Ocan Sankur","year":"2017","venue":"Acta Inf","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Cimatti et al., 1998] Alessandro Cimatti, Marco Roveri, and Paolo Traverso. Strong planning in non-deterministic domains via model checking. In AIPS, pages 36-43. AAAI, 1998.","title":"Strong planning in non-deterministic domains via model checking","authors":"Alessandro Cimatti, Marco Roveri, Paolo Traverso","year":"1998","venue":"AIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Cimatti et al., 2003] Alessandro Cimatti, Marco Pistore, Marco Roveri, and Paolo Traverso. Weak, strong, and strong cyclic planning via symbolic model checking. Arti- ficial Intelligence, 1-2(147), 2003.","title":"Weak, strong, and strong cyclic planning via symbolic model checking","authors":"Alessandro Cimatti, Marco Pistore, Marco Roveri, Paolo Traverso","year":"2003","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Daniele et al., 1999] Marco Daniele, Paolo Traverso, and Moshe Y. Vardi. Strong cyclic planning revisited. In ECP, 1999.","title":"Strong cyclic planning revisited","authors":"Marco Daniele, Paolo Traverso, Moshe Y Vardi","year":"1999","venue":"ECP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[De Giacomo and Vardi, 2013] Giuseppe De Giacomo and Moshe Y. Vardi. Linear temporal logic and linear dynamic logic on finite traces. In IJCAI, 2013.","title":"Linear temporal logic and linear dynamic logic on finite traces","authors":"De Giacomo, Giuseppe De Vardi, Moshe Y Giacomo, Vardi","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/132"},{"position":21,"raw":"[D'Ippolito et al., 2018] Nicolás D'Ippolito, Natalia Rodríguez, and Sebastian Sardiña. Fully observable non-deterministic planning as assumption-based reactive synthesis. J. Artif. Intell. Res., 61:593-621, 2018.","title":"Fully observable non-deterministic planning as assumption-based reactive synthesis","authors":"D' Nicolás, Natalia Ippolito, Sebastian Rodríguez, Sardiña","year":"2018","venue":"J. Artif. Intell. Res","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Faella, 2009] Marco Faella. Admissible strategies in infinite games over graphs. In MFCS, 2009.","title":"Admissible strategies in infinite games over graphs","authors":"Marco Faella, Faella","year":"2009","venue":"MFCS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Geffner and Bonet, 2013] H. Geffner and B. Bonet. A Co- incise Introduction to Models and Methods for Automated Planning. M&C, 2013.","title":"A Coincise Introduction to Models and Methods for Automated Planning","authors":"H Geffner, B Bonet","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Geffner and Geffner, 2018] Tomas Geffner and Hec- tor Geffner. Compact policies for fully observable non-deterministic planning as SAT. In ICAPS, 2018.","title":"Compact policies for fully observable non-deterministic planning as SAT","authors":"Geffner ; Tor Geffner, Geffner","year":"2018","venue":"ICAPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ghallab et al., 2016] Malik Ghallab, Dana S. Nau, and Paolo Traverso. Automated planning and action. CUP, 2016.","title":"Automated planning and action","authors":"Malik Ghallab, Dana S Nau, Paolo Traverso","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Haslum et al., 2019] Patrik Haslum, Nir Lipovetzky, Daniele Magazzeni, and Christian Muise. An Introduction to the Planning Domain Definition Language. M&C, 2019.","title":"An Introduction to the Planning Domain Definition Language","authors":"Haslum","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Hu and De Giacomo, 2011] Yuxiao Hu and Giuseppe De Giacomo. Generalized planning: Synthesizing plans that work for multiple environments. In IJCAI, 2011.","title":"Generalized planning: Synthesizing plans that work for multiple environments","authors":"De Hu, ; Yuxiao Giacomo, Giuseppe Hu, Giacomo De","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2011/159"},{"position":28,"raw":"[Jensen et al., 2001] Rune M. Jensen, Manuela M. Veloso, and Michael H. Bowling. OBDD-based optimistic and strong cyclic adversarial planning. In ECP, 2001.","title":"OBDD-based optimistic and strong cyclic adversarial planning","authors":"M Rune, Manuela M Jensen, Michael H Veloso, Bowling","year":"2001","venue":"ECP","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Kiefer et al., 2020] Stefan Kiefer, Richard Mayr, Mahsa Shirmohammadi, and Patrick Totzke. Strategy complex- ity of parity objectives in countable mdps. In CONCUR, 2020.","title":"Strategy complexity of parity objectives in countable mdps","authors":"Stefan Kiefer, Richard Mayr, Mahsa Shirmohammadi, Patrick Totzke","year":"2020","venue":"CONCUR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Kissmann and Edelkamp, 2011] Peter Kissmann and Stefan Edelkamp. Gamer, a general game playing agent. Künst. Intell., 25(1):49-52, 2011.","title":"Gamer, a general game playing agent","authors":"Peter Kissmann, Stefan Edelkamp","year":"2011","venue":"Künst. Intell","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Mattmüller et al., 2010] Robert Mattmüller, Manuela Or- tlieb, Malte Helmert, and Pascal Bercher. Pattern database heuristics for fully observable nondeterministic planning. In ICAPS, 2010.","title":"Pattern database heuristics for fully observable nondeterministic planning","authors":"Robert Mattmüller, Manuela Ortlieb, Malte Helmert, Pascal Bercher","year":"2010","venue":"ICAPS","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Muise et al., 2012] Christian J. Muise, Sheila A. McIlraith, and J. Christopher Beck. Improved non-deterministic planning by exploiting state relevance. In ICAPS, 2012.","title":"Improved non-deterministic planning by exploiting state relevance","authors":"Christian J Muise, Sheila A Mcilraith, J Christopher Beck","year":"2012","venue":"ICAPS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Pistore and Traverso, 2001] Marco Pistore and Paolo Traverso. Planning as model checking for extended goals in non-deterministic domains. In IJCAI, 2001.","title":"Planning as model checking for extended goals in non-deterministic domains","authors":"Traverso Pistore, Marco Pistore, Paolo Traverso","year":"2001","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2001/67"},{"position":34,"raw":"[Puterman, 2005] Martin L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, 2005.","title":"Markov Decision Processes: Discrete Stochastic Dynamic Programming","authors":"Martin L Puterman","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Ramírez and Sardiña, 2014] Miquel Ramírez and Sebastian Sardiña. Directed fixed-point regression-based planning for non-deterministic domains. In ICAPS, 2014.","title":"Directed fixed-point regression-based planning for non-deterministic domains","authors":"Miquel Ramírez, Sebastian Sardiña","year":"2014","venue":"ICAPS","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Rintanen, 2004] Jussi Rintanen. Complexity of planning with partial observability. In ICAPS, 2004.","title":"Complexity of planning with partial observability","authors":"Jussi Rintanen","year":"2004","venue":"ICAPS","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Srivastava, 2010] Siddharth Srivastava. Foundations and Applications of Generalized Planning. PhD thesis, Uni- versity of Massachusetts Amherst, 2010.","title":"Foundations and Applications of Generalized Planning","authors":null,"year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Vardi and Wolper, 1986] Moshe Y. Vardi and Pierre Wolper. An automata-theoretic approach to automatic program ver- ification. In LICS, 1986.","title":"An automata-theoretic approach to automatic program verification","authors":"Moshe Y Vardi, Pierre Wolper","year":"1986","venue":"LICS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}