{"type":"ScholarlyArticle","title":"The Trembling-Hand Problem for LTLf Planning","authors":[{"name":"Pian Yu","slug":"pian-yu","given":"Pian","family":"Yu","affiliation":"Department of Computer Science , University of Oxford , UK"},{"name":"Shufang Zhu","slug":"shufang-zhu","given":"Shufang","family":"Zhu","affiliation":"Department of Computer Science , University of Oxford , UK"},{"name":"Giuseppe De Giacomo","slug":"giuseppe-de-giacomo","given":"Giuseppe","family":"De Giacomo","affiliation":"Department of Computer Science , University of Oxford , UK"},{"name":"Marta Kwiatkowska","slug":"marta-kwiatkowska","given":"Marta","family":"Kwiatkowska","affiliation":"Department of Computer Science , University of Oxford , UK"},{"name":"Moshe Vardi","slug":"moshe-vardi","given":"Moshe","family":"Vardi","affiliation":"Department of Computer Science , Rice University , USA"}],"abstract":"Consider an agent acting to achieve its temporal goal, but with a ``trembling hand\". In this case, the agent may mistakenly instruct, with a certain (typically small) probability, actions that are not intended due to faults or imprecision in its action selection mechanism, thereby leading to possible goal failure. We study the trembling-hand problem in the context of reasoning about actions and planning for temporally extended goals expressed in Linear Temporal Logic on finite traces (LTLf), where we want to synthesize a strategy (aka plan) that maximizes the probability of satisfying the LTLf goal in spite of the trembling hand. We consider both deterministic and nondeterministic (adversarial) domains. We propose solution techniques for both cases by relying respectively on Markov Decision Processes and on Markov Decision Processes with Set-valued Transitions with LTLf objectives, where the set-valued probabilistic transitions capture both the nondeterminism from the environment and the possible action instruction errors from the agent. We formally show the correctness of our solution techniques and demonstrate their effectiveness experimentally through a proof-of-concept implementation.","doi":"10.24963/ijcai.2024/402","url":"https://ijcai.org/proceedings/2024/402","pdf":"https://ijcai.org/proceedings/2024/0402.pdf","markdown":"https://ijcai.org/proceedings/2024/402.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/402","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Knowledge Representation and Reasoning","track":"Main Track","number":402,"pages":{"first":3631,"last":3641},"keywords":["Planning and Scheduling: PS: Markov decisions processes","Knowledge Representation and Reasoning: KRR: Reasoning about actions","Agent-based and Multi-agent Systems: MAS: Formal verification, validation and synthesis"],"videos":[],"references":[{"position":1,"raw":"[Aineto et al., 2023] Diego Aineto, Alessandro Gaudenzi, Alfonso Gerevini, Alberto Rovetta, Enrico Scala, and Ivan Serina. Action-failure resilient planning. In ECAI, pages 44-51, 2023.","title":"Action-failure resilient planning","authors":"Aineto","year":"2023","venue":"ECAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Altman, 1998] Eitan Altman. Constrained Markov decision processes with total cost criteria: Lagrangian approach and dual linear program. Mathematical methods of operations research, 48(3):387-417, 1998.","title":"Constrained Markov decision processes with total cost criteria: Lagrangian approach and dual linear program","authors":"Eitan Altman","year":"1998","venue":"Mathematical methods of operations research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Aminof et al., 2021] Benjamin Aminof, Giuseppe De Gia- como, Alessio Lomuscio, Aniello Murano, and Sasha Ru- bin. Synthesizing best-effort strategies under multiple en- vironment specifications. In KR, pages 42-51, 2021.","title":"Synthesizing best-effort strategies under multiple environment specifications","authors":"Benjamin Aminof, Giuseppe De Giacomo, Alessio Lomuscio, Aniello Murano, Sasha Rubin","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bacchus et al., 1999] Fahiem Bacchus, Joseph Y Halpern, and Hector J Levesque. Reasoning about noisy sensors and effectors in the situation calculus. Artificial Intelligence, 111(1-2):171-208, 1999.","title":"Reasoning about noisy sensors and effectors in the situation calculus","authors":"Bacchus","year":"1999","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Baier and Katoen, 2008] Christel Baier and Joost-Pieter Katoen. Principles of model checking. MIT press, 2008.","title":"Principles of model checking","authors":"Christel Baier, Joost-Pieter Katoen","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"R. Selten Bielefeld. Reexamination of the Perfectness Concept for Equilibrium Points in Extensive Games, pages 1-31. Dordrecht, 1988.","title":"Reexamination of the Perfectness Concept for Equilibrium Points in Extensive Games","authors":"R, Selten Bielefeld","year":"1988","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bonassi et al., 2023] Luigi Bonassi, Giuseppe De Giacomo, Marco Favorito, Francesco Fuggitti, Alfonso Emilio Gerevini, and Enrico Scala. Planning for temporally ex- tended goals in pure-past linear temporal logic. In ICAPS, pages 61-69, 2023.","title":"Planning for temporally extended goals in pure-past linear temporal logic","authors":"Luigi Bonassi, Giuseppe De Giacomo, Marco Favorito, Francesco Fuggitti, Alfonso Emilio Gerevini, Enrico Scala","year":"2023","venue":"ICAPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Brafman et al., 2018] Ronen I. Brafman, Giuseppe De Gia- como, and Fabio Patrizi. LTL f /LDL f non-markovian re- wards. In Sheila A. McIlraith and Kilian Q. Weinberger, editors, AAAI, pages 1771-1778, 2018.","title":"LTL f /LDL f non-markovian rewards","authors":"Ronen I Brafman, Giuseppe De, Giacomo, Fabio Patrizi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Buffet et al., 2005] Olivier Buffet, Douglas Aberdeen, et al. Robust planning with (l) rtdp. In INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLI- GENCE, volume 19, page 1214.","title":"Robust planning with (l) rtdp","authors":"Olivier Buffet, Douglas Aberdeen","year":null,"venue":"INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLI-GENCE","doi":null,"url":null,"internal_url":"/proceedings/2005/219"},{"position":10,"raw":"LAWRENCE ERL- BAUM ASSOCIATES LTD, 2005.","title":null,"authors":"Erl-Baum Associates Lawrence, Ltd","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Cimatti et al., 2003] Alessandro Cimatti, Marco Pistore, Marco Roveri, and Paolo Traverso. Weak, strong, and strong cyclic planning via symbolic model checking. Artif. Intell., 147(1-2):35-84, 2003.","title":"Weak, strong, and strong cyclic planning via symbolic model checking","authors":"Alessandro Cimatti, Marco Pistore, Marco Roveri, Paolo Traverso","year":"2003","venue":"Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ciolek et al., 2020] Daniel Alfredo Ciolek, Nicolás D'Ippolito, Alberto Pozanco, and Sebastian Sardiña. Multi-tier automated planning for adaptive behavior. In ICAPS, pages 66-74. AAAI Press, 2020.","title":"Multi-tier automated planning for adaptive behavior","authors":"Alfredo Daniel, Ciolek, D' Nicolás, Alberto Ippolito, Sebastian Pozanco, Sardiña","year":"2020","venue":"ICAPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[De Giacomo and Favorito, 2021] Giuseppe De Giacomo and Marco Favorito. Compositional approach to translate LTL f /LDL f into deterministic finite automata. In ICAPS, pages 122-130, 2021.","title":"Compositional approach to translate LTL f /LDL f into deterministic finite automata","authors":"De Giacomo, Giuseppe Favorito, Giacomo De, Marco Favorito","year":"2021","venue":"ICAPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[De Giacomo and Rubin, 2018] Giuseppe De Giacomo and Sasha Rubin. Automata-theoretic foundations of FOND planning for LTL f and LDL f goals. In IJCAI, pages 4729- 4735, 2018.","title":"Automata-theoretic foundations of FOND planning for LTL f and LDL f goals","authors":"De Giacomo, Rubin, Giuseppe De, Giacomo, Sasha Rubin","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[De Giacomo and Vardi, 2013] Giuseppe De Giacomo and Moshe Y. Vardi. Linear Temporal Logic and Linear Dy- namic Logic on Finite Traces. In IJCAI, 2013.","title":"Linear Temporal Logic and Linear Dynamic Logic on Finite Traces","authors":"De Giacomo, Giuseppe De Vardi, Moshe Y Giacomo, Vardi","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/132"},{"position":16,"raw":"[De Giacomo et al., 2023] Giuseppe De Giacomo, Gian- marco Parretti, and Shufang Zhu. LTL f best-effort synthe- sis in nondeterministic planning domains. pages 533-540, 2023.","title":"LTL f best-effort synthesis in nondeterministic planning domains","authors":"Giuseppe De, Giacomo, Gianmarco Parretti, Shufang Zhu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Geffner and Bonet, 2013] Hector Geffner and Blai Bonet. A Concise Introduction to Models and Methods for Auto- mated Planning. Morgan & Claypool Publishers, 2013.","title":"A Concise Introduction to Models and Methods for Automated Planning","authors":"Bonet Geffner, Hector Geffner, Blai Bonet","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ghallab et al., 2004] Malik Ghallab, Dana S. Nau, and Paolo Traverso. Automated planning -theory and prac- tice. 2004.","title":"Automated planning -theory and practice","authors":"Malik Ghallab, Dana S Nau, Paolo Traverso","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Givan et al., 2000] Robert Givan, Sonia Leach, and Thomas Dean. Bounded-parameter Markov decision processes. Ar- tificial Intelligence, 122(1-2):71-109, 2000.","title":"Bounded-parameter Markov decision processes","authors":"Robert Givan, Sonia Leach, Thomas Dean","year":"2000","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Hahn et al., 2019] Ernst Moritz Hahn, Vahid Hashemi, Hol- ger Hermanns, Morteza Lahijanian, and Andrea Tur- rini. Interval Markov decision processes with multi- ple objectives: from robust strategies to Pareto curves. ACM Transactions on Modeling and Computer Simulation (TOMACS), 29(4):1-31, 2019.","title":"Interval Markov decision processes with multiple objectives: from robust strategies to Pareto curves","authors":"Hahn","year":"2019","venue":"ACM Transactions on Modeling and Computer Simulation (TOMACS)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Haslum et al., 2019] Patrik Haslum, Nir Lipovetzky, Daniele Magazzeni, and Christian Muise. An Introduction to the Planning Domain Definition Language. 2019.","title":"An Introduction to the Planning Domain Definition Language","authors":"Haslum","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[He et al., 2019] Keliang He, Andrew M Wells, Lydia E Kavraki, and Moshe Y Vardi. Efficient symbolic reactive synthesis for finite-horizon tasks. In 2019 International Conference on Robotics and Automation (ICRA), pages 8993-8999, 2019.","title":"Efficient symbolic reactive synthesis for finite-horizon tasks","authors":"He","year":"2019","venue":"2019 International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Marchesi and Gatti, 2021] Alberto Marchesi and Nicola Gatti. Trembling-hand perfection and correlation in se- quential games. In AAAI, pages 5566-5574, 2021.","title":"Trembling-hand perfection and correlation in sequential games","authors":"Alberto Marchesi, Nicola Gatti","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Nilim and El Ghaoui, 2004] Arnab Nilim and Laurent El Ghaoui. Robust markov decision processes with uncertain transition matrices. PhD thesis, University of California, Berkeley, 2004.","title":"Robust markov decision processes with uncertain transition matrices","authors":"Arnab Nilim, Laurent El Ghaoui","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Pnueli, 1977] Amir Pnueli. The temporal logic of programs. In FOCS, pages 46-57, 1977.","title":"The temporal logic of programs","authors":"Amir Pnueli, Pnueli","year":"1977","venue":"FOCS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Puterman, 2014] Martin L Puterman. Markov decision pro- cesses: discrete stochastic dynamic programming. 2014.","title":"Markov decision processes: discrete stochastic dynamic programming","authors":"Martin L Puterman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Reiter, 2001] Raymond Reiter. Knowledge in Action: Logi- cal Foundations for Specifying and Implementing Dynam- ical Systems. MIT, 2001.","title":"Knowledge in Action: Logical Foundations for Specifying and Implementing Dynamical Systems","authors":"Raymond Reiter, Reiter","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Satia and Lave Jr, 1973] Jay K Satia and Roy E Lave Jr. Markovian decision processes with uncertain transition probabilities. Operations Research, 21(3):728-740, 1973.","title":"Markovian decision processes with uncertain transition probabilities","authors":"Lave Satia, ; ] Jr, K Jay, Roy E Lave Satia, Jr","year":"1973","venue":"Operations Research","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Trevizan et al., 2007] Felipe W Trevizan, Fabio Gagliardi Cozman, and Leliane Nunes de Barros. Planning under risk and knightian uncertainty. In IJCAI, pages 2023- 2028, 2007.","title":"Planning under risk and knightian uncertainty","authors":"Fabio Felipe W Trevizan, Leliane Gagliardi Cozman, Nunes De Barros","year":"2007","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2007/326"},{"position":30,"raw":"[Trevizan et al., 2008] Felipe W Trevizan, Fábio G Cozman, and Leliane N De Barros. Mixed probabilistic and non- deterministic factored planning through markov decision processes with set-valued transitions. In Workshop on A Reality Check for Planning and Scheduling Under Uncer- tainty at ICAPS, page 62, 2008.","title":"Mixed probabilistic and nondeterministic factored planning through markov decision processes with set-valued transitions","authors":"Felipe W Trevizan, Leliane N De Fábio G Cozman, Barros","year":"2008","venue":"Workshop on A Reality Check for Planning and Scheduling Under Uncertainty at ICAPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Vardi, 2020] Moshe Y. Vardi. Efficiency vs. resilience: what COVID-19 teaches computing. Commun. ACM, 63(5):9, 2020.","title":"Efficiency vs. resilience: what COVID-19 teaches computing","authors":"Moshe Y Vardi","year":"2020","venue":"Commun. ACM","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wells et al., 2020a] Andrew M. Wells, Morteza Lahijanian, Lydia E. Kavraki, and Moshe Y. Vardi. LTL f synthesis on probabilistic systems. In GandALF, volume 326 of EPTCS, pages 166-181, 2020.","title":"LTL f synthesis on probabilistic systems","authors":"Wells","year":"2020","venue":"GandALF","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wells et al., 2020b] Andrew M. Wells, Morteza Lahijanian, Lydia E. Kavraki, and Moshe Y. Vardi. Ltl f synthesis on probabilistic systems. In GandALF, volume 326 of EPTCS, pages 166-181, 2020.","title":"Ltl f synthesis on probabilistic systems","authors":"Wells","year":"2020","venue":"GandALF","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wells et al., 2021] Andrew M. Wells, Zachary K. Kingston, Morteza Lahijanian, Lydia E. Kavraki, and Moshe Y. Vardi. Finite-horizon synthesis for probabilistic manipu- lation domains. In ICRA, pages 6336-6342, 2021.","title":"Finite-horizon synthesis for probabilistic manipulation domains","authors":"Andrew M Wells, Zachary K Kingston, Morteza Lahijanian, Lydia E Kavraki, Moshe Y Vardi","year":"2021","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[White III and Eldeib, 1994] Chelsea C White III and Hany K Eldeib. Markov decision processes with im- precise transition probabilities. Operations Research, 42(4):739-749, 1994.","title":"Markov decision processes with imprecise transition probabilities","authors":"Iii White, Chelsea C Eldeib, Iii White, K Hany, Eldeib","year":"1994","venue":"Operations Research","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhu and De Giacomo, 2022] Shufang Zhu and Giuseppe De Giacomo. Synthesis of maximally permissive strategies for LTL f specifications. In IJCAI, pages 2783-2789, 2022.","title":"Synthesis of maximally permissive strategies for LTL f specifications","authors":"Shufang Zhu, Giuseppe De, Giacomo","year":"2022","venue":"IJCAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}