{"type":"ScholarlyArticle","title":"Probabilistic Planning with Prioritized Preferences over Temporal Logic Objectives","authors":[{"name":"Lening Li","slug":"lening-li","given":"Lening","family":"Li","affiliation":"Department of Robotics Engineering , Worcester Polytechnic Institute , Worcester , MA 01609 , USA."},{"name":"Hazhar Rahmani","slug":"hazhar-rahmani","given":"Hazhar","family":"Rahmani","affiliation":"Department of Electrical and Computer Engineering , University of Florida , Gainesville , FL 32605 , USA."},{"name":"Jie Fu","slug":"jie-fu","given":"Jie","family":"Fu","affiliation":"Department of Electrical and Computer Engineering , University of Florida , Gainesville , FL 32605 , USA."}],"abstract":"This paper studies temporal planning in probabilistic environments, modeled as labeled Markov decision processes (MDPs), with user preferences over multiple temporal goals.  Existing works reflect such preferences as a prioritized list of goals. This paper introduces a new specification language, termed prioritized qualitative choice linear temporal logic on finite traces, which augments linear temporal logic on finite traces with prioritized conjunction and ordered disjunction from prioritized qualitative choice logic. This language allows for succinctly specifying temporal objectives with corresponding preferences accomplishing each temporal task. The finite traces that describe the system's behaviors are ranked based on their dissatisfaction scores with respect to the formula. We propose a systematic translation from the new language to a weighted deterministic finite automaton. Utilizing this computational model, we formulate and solve a problem of computing an optimal policy that minimizes the expected score of dissatisfaction given user preferences. We demonstrate the efficacy and applicability of the logic and the algorithm on several case studies with detailed analyses for each.","doi":"10.24963/ijcai.2023/22","url":"https://ijcai.org/proceedings/2023/22","pdf":"https://ijcai.org/proceedings/2023/0022.pdf","markdown":"https://ijcai.org/proceedings/2023/22.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/22","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":22,"pages":{"first":189,"last":198},"keywords":["Planning and Scheduling: PS: Markov decisions processes","Knowledge Representation and Reasoning: KRR: Preference modelling and preference-based reasoning","Agent-based and Multi-agent Systems: MAS: Formal verification, validation and synthesis"],"videos":[],"references":[{"position":1,"raw":"[Benferhat and Sedki, 2007] Salem Benferhat and Karima Sedki. A revised qualitative choice logic for handling prioritized preferences. In European Conference on Sym- bolic and Quantitative Approaches to Reasoning and Un- certainty, pages 635-647. Springer, 2007.","title":"Salem Benferhat and Karima Sedki. A revised qualitative choice logic for handling prioritized preferences","authors":"Sedki Benferhat","year":"2007","venue":"European Conference on Symbolic and Quantitative Approaches to Reasoning and Uncertainty","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Benferhat and Sedki, 2008] Salem Benferhat and Karima Sedki. Two alternatives for handling preferences in qualitative choice logic. Fuzzy Sets and Systems, 159(15):1889-1912, August 2008.","title":"Salem Benferhat and Karima Sedki. Two alternatives for handling preferences in qualitative choice logic","authors":"Sedki Benferhat","year":"2008","venue":"Fuzzy Sets and Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bertsekas and Yu, 2013] Dimitri P Bertsekas and Huizhen Yu. Stochastic shortest path problems under weak condi- tions. Lab. for Information and Decision Systems Report LIDS-P-2909, MIT, 2013.","title":"Stochastic shortest path problems under weak conditions","authors":"P Dimitri, Huizhen Bertsekas, Yu","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bradley et al., 2021] Christopher Bradley, Adam Pacheck, Gregory J Stein, Sebastian Castro, Hadas Kress-Gazit, and Nicholas Roy. Learning and planning for tempo- rally extended tasks in unknown environments. In 2021 IEEE International Conference on Robotics and Automa- tion (ICRA), pages 4830-4836. IEEE, 2021.","title":"Learning and planning for temporally extended tasks in unknown environments","authors":"Christopher Bradley, Adam Pacheck, Gregory J Stein, Sebastian Castro, Hadas Kress-Gazit, Nicholas Roy","year":"2021","venue":"2021 IEEE International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Brewka et al., 2004] Gerhard Brewka, Salem Benferhat, and Daniel Le Berre. Qualitative choice logic. Artificial Intelligence, 157(1):203-237, August 2004.","title":"Qualitative choice logic","authors":"Gerhard Brewka, Salem Benferhat, Daniel Le Berre","year":"2004","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cai et al., 2020] Mingyu Cai, Hao Peng, Zhijun Li, Hongbo Gao, and Zhen Kan. Receding horizon control-based mo- tion planning with partially infeasible ltl constraints. IEEE Control Systems Letters, 5(4):1279-1284, 2020.","title":"Receding horizon control-based motion planning with partially infeasible ltl constraints","authors":"Mingyu Cai, Hao Peng, Zhijun Li, Hongbo Gao, Zhen Kan","year":"2020","venue":"IEEE Control Systems Letters","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Camacho et al., 2017] Alberto Camacho, Eleni Triantafil- lou, Christian Muise, Jorge A Baier, and Sheila A McIl- raith. Non-deterministic planning with temporally ex- tended goals: Ltl over finite and infinite traces. In Thirty- First AAAI Conference on Artificial Intelligence, 2017.","title":"Non-deterministic planning with temporally extended goals: Ltl over finite and infinite traces","authors":"Alberto Camacho, Eleni Triantafillou, Christian Muise, Jorge A Baier, Sheila A Mcilraith","year":"2017","venue":"Thirty-First AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[De Giacomo and Favorito, 2021] Giuseppe De Giacomo and Marco Favorito. Compositional approach to translate ltlf/ldlf into deterministic finite automata. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 31, pages 122-130, 2021.","title":"Compositional approach to translate ltlf/ldlf into deterministic finite automata","authors":"De Giacomo, Giuseppe Favorito, Giacomo De, Marco Favorito","year":"2021","venue":"Proceedings of the International Conference on Automated Planning and Scheduling","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[De Giacomo and Vardi, 2013] Giuseppe De Giacomo and Moshe Y Vardi. Linear temporal logic and linear dynamic logic on finite traces. In IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence, pages 854-860. Association for Computing Machinery, 2013.","title":"Linear temporal logic and linear dynamic logic on finite traces","authors":"De Giacomo, Giuseppe Vardi, Giacomo De, Moshe Y Vardi","year":"2013","venue":"IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[De Giacomo and Vardi, 2015] Giuseppe De Giacomo and Moshe Vardi. Synthesis for ltl and ldl on finite traces. In Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015.","title":"Giuseppe De Giacomo and Moshe Vardi. Synthesis for ltl and ldl on finite traces","authors":"De Giacomo, Vardi","year":"2015","venue":"Twenty-Fourth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Droste and Gastin, 2009] Manfred Droste and Paul Gastin. Weighted automata and weighted logics. In Handbook of weighted automata, pages 175-211. Springer, 2009.","title":"Weighted automata and weighted logics","authors":"Gastin Droste, Manfred Droste, Paul Gastin","year":"2009","venue":"Handbook of weighted automata","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2015] Keliang He, Morteza Lahijanian, Lydia E Kavraki, and Moshe Y Vardi. Towards manipulation planning with temporal logic specifications. In 2015 IEEE international conference on robotics and automation (ICRA), pages 346-352. IEEE, 2015.","title":"Towards manipulation planning with temporal logic specifications","authors":"He","year":"2015","venue":"IEEE international conference on robotics and automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2020] Binghan He, Jaemin Lee, Ufuk Topcu, and Luis Sentis. Bp-rrt: Barrier pair synthesis for temporal logic motion planning. In 2020 59th IEEE Conference on Decision and Control (CDC), pages 1404-1409. IEEE, 2020.","title":"Bp-rrt: Barrier pair synthesis for temporal logic motion planning","authors":"He","year":"2020","venue":"59th IEEE Conference on Decision and Control (CDC)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hekmatnejad and Fainekos, 2018] Mohammad Hekmatne- jad and Georgios Fainekos. Optimal multi-valued ltl plan- ning for systems with access right levels. In 2018 Annual American Control Conference (ACC), pages 2363-2370. IEEE, 2018.","title":"Optimal multi-valued ltl planning for systems with access right levels","authors":"Mohammad Hekmatnejad, Georgios Fainekos","year":"2018","venue":"Annual American Control Conference (ACC)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kantaros et al., 2022] Yiannis Kantaros, Samarth Kallu- raya, Qi Jin, and George J Pappas. Perception-based tem- poral logic planning in uncertain semantic maps. IEEE Transactions on Robotics, 2022.","title":"Perception-based temporal logic planning in uncertain semantic maps","authors":"Yiannis Kantaros, Samarth Kalluraya, Jin Qi, George J Pappas","year":"2022","venue":"IEEE Transactions on Robotics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kasenberg et al., 2020] Daniel Kasenberg, Ravenna Thiel- strom, and Matthias Scheutz. Generating explanations for temporal logic planner decisions. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 30, pages 449-458, 2020.","title":"Generating explanations for temporal logic planner decisions","authors":"Daniel Kasenberg, Ravenna Thielstrom, Matthias Scheutz","year":"2020","venue":"Proceedings of the International Conference on Automated Planning and Scheduling","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lacerda et al., 2015] Bruno Lacerda, David Parker, and Nick Hawes. Optimal policy generation for partially satis- fiable co-safe ltl specifications. In Twenty-Fourth Interna- tional Joint Conference on Artificial Intelligence, 2015.","title":"Optimal policy generation for partially satisfiable co-safe ltl specifications","authors":"Lacerda","year":"2015","venue":"Twenty-Fourth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lahijanian and Kwiatkowska, 2016] Morteza Lahijanian and Marta Kwiatkowska. Specification revision for Markov decision processes with optimal trade-off. In Proc. 55th Conference on Decision and Control (CDC'16), pages 7411-7418, 2016.","title":"Specification revision for Markov decision processes with optimal trade-off","authors":"Lahijanian, Kwiatkowska, Morteza Lahijanian, Marta Kwiatkowska","year":"2016","venue":"Proc. 55th Conference on Decision and Control (CDC'16)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2020] Meilun Li, Andrea Turrini, Ernst Moritz Hahn, Zhikun She, and Lijun Zhang. Probabilistic pref- erence planning problem for markov decision processes. IEEE transactions on software engineering, 2020.","title":"Probabilistic preference planning problem for markov decision processes","authors":"Li","year":"2020","venue":"IEEE transactions on software engineering","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2021] Shen Li, Daehyung Park, Yoonchang Sung, Julie A Shah, and Nicholas Roy. Reactive task and mo- tion planning under temporal logic specifications. In 2021 IEEE International Conference on Robotics and Automa- tion (ICRA), pages 12618-12624. IEEE, 2021.","title":"Reactive task and motion planning under temporal logic specifications","authors":"Shen Li, Daehyung Park, Yoonchang Sung, Julie A Shah, Nicholas Roy","year":"2021","venue":"2021 IEEE International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mallett et al., 2021] Ian Mallett, Sylvie Thiébaux, and Fe- lipe Trevizan. Progression heuristics for planning with probabilistic ltl constraints. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11870-11879, 2021.","title":"Progression heuristics for planning with probabilistic ltl constraints","authors":"Ian Mallett, Sylvie Thiébaux, Felipe Trevizan","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Mehdipour et al., 2021] Noushin Mehdipour, Cristian-Ioan Vasile, and Calin Belta. Specifying User Preferences Us- ing Weighted Signal Temporal Logic. IEEE Control Sys- tems Letters, 5(6):2006-2011, December 2021.","title":"Specifying User Preferences Using Weighted Signal Temporal Logic","authors":"Noushin Mehdipour, Cristian-Ioan Vasile, Calin Belta","year":"2021","venue":"IEEE Control Systems Letters","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Pnueli, 1981] Amir Pnueli. The temporal semantics of concurrent programs. Theoretical computer science, 13(1):45-60, 1981.","title":"The temporal semantics of concurrent programs","authors":"Amir Pnueli, Pnueli","year":"1981","venue":"Theoretical computer science","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Puterman, 2014] Martin L Puterman. Markov decision pro- cesses: discrete stochastic dynamic programming. John Wiley & Sons, 2014.","title":"Markov decision processes: discrete stochastic dynamic programming","authors":"Martin L Puterman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Rahmani and O'Kane, 2019] Hazhar Rahmani and Jason M O'Kane. Optimal temporal logic planning with cascad- ing soft constraints. In 2019 IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS), pages 2524-2531. IEEE, 2019.","title":"Optimal temporal logic planning with cascading soft constraints","authors":"O'kane ; Hazhar Rahmani, Jason M O' Rahmani, Kane","year":"2019","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Rahmani and O'Kane, 2020] Hazhar Rahmani and Jason M O'Kane. What to do when you can't do it all: Tem- poral logic planning with soft temporal logic constraints. In 2020 IEEE/RSJ International Conference on Intelli- gent Robots and Systems (IROS), pages 6619-6626. IEEE, 2020.","title":"What to do when you can't do it all: Temporal logic planning with soft temporal logic constraints","authors":"O'kane ; Hazhar Rahmani, Jason M O' Rahmani, Kane","year":"2020","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tumova et al., 2013a] Jana Tumova, Luis I Reyes Cas- tro, Sertac Karaman, Emilio Frazzoli, and Daniela Rus. Minimum-violation ltl planning with conflicting specifica- tions. In American Control Conference, pages 200-205. IEEE, 2013.","title":"Minimum-violation ltl planning with conflicting specifications","authors":"Tumova","year":"2013","venue":"American Control Conference","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tumova et al., 2013b] Jana Tumova, Gavin C Hall, Ser- tac Karaman, Emilio Frazzoli, and Daniela Rus. Least- violating control strategy synthesis with safety rules. In Proc. Int. Conf. on Hybrid systems: Computation and con- trol, 2013.","title":"Leastviolating control strategy synthesis with safety rules","authors":"Tumova","year":"2013","venue":"Proc. Int. Conf. on Hybrid systems: Computation and control","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Vasile et al., 2017] Cristian-Ioan Vasile, Jana Tumova, Ser- tac Karaman, Calin Belta, and Daniela Rus. Minimum- violation scltl motion planning for mobility-on-demand. pages 1481-1488. IEEE, 2017.","title":"Minimumviolation scltl motion planning for mobility-on-demand","authors":"Cristian-Ioan Vasile, Jana Tumova, Sertac Karaman, Calin Belta, Daniela Rus","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Vasile et al., 2020] Cristian Ioan Vasile, Xiao Li, and Calin Belta. Reactive sampling-based path planning with tem- poral logic specifications. The International Journal of Robotics Research, 39(8):1002-1028, 2020.","title":"Reactive sampling-based path planning with temporal logic specifications","authors":"Cristian Ioan Vasile, Xiao Li, Calin Belta","year":"2020","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2020] Yu Wang, Siddhartha Nalluri, and Miroslav Pajic. Hyperproperties for robotics: Planning via hyperltl. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 8462-8468. IEEE, 2020.","title":"Hyperproperties for robotics: Planning via hyperltl","authors":"Yu Wang, Siddhartha Nalluri, Miroslav Pajic","year":"2020","venue":"IEEE International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wolper, 2001] Pierre Wolper. Constructing Automata from Temporal Logic Formulas: A Tutorial. In G. Goos, J. Hartmanis, J. van Leeuwen, Ed Brinksma, Holger Her- manns, and Joost-Pieter Katoen, editors, Lectures on For- mal Methods and PerformanceAnalysis, volume 2090, pages 261-277. Springer Berlin Heidelberg, Berlin, Hei- delberg, 2001.","title":"Constructing Automata from Temporal Logic Formulas: A Tutorial","authors":"Pierre Wolper, Wolper","year":"2001","venue":"Lectures on Formal Methods and PerformanceAnalysis","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wongpiromsarn et al., 2021] Tichakorn Wongpiromsarn, Konstantin Slutsky, Emilio Frazzoli, and Ufuk Topcu. Minimum-violation planning for autonomous systems: Theoretical and practical considerations. In 2021 American Control Conference, 2021.","title":"Minimum-violation planning for autonomous systems: Theoretical and practical considerations","authors":"Wongpiromsarn","year":"2021","venue":"2021 American Control Conference","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yang et al., 2020] Yuanjiang Yang, Xiang Yin, and Shaoyuan Li. A distributed framework for multi-robot task planning with temporal logic specifications. In 2020 IEEE 16th International Conference on Control & Automation (ICCA), pages 570-575. IEEE, 2020.","title":"A distributed framework for multi-robot task planning with temporal logic specifications","authors":"Yuanjiang Yang, Xiang Yin, Shaoyuan Li","year":"2020","venue":"IEEE 16th International Conference on Control & Automation (ICCA)","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhao et al., 2022] Jiawei Zhao, Xiang Yin, and Shaoyuan Li. Temporal logic robot task planning with active acquisi- tion of information. In 2022 IEEE 17th International Con- ference on Control & Automation (ICCA), pages 1014- 1020. IEEE, 2022.","title":"Temporal logic robot task planning with active acquisition of information","authors":"Jiawei Zhao, Xiang Yin, Shaoyuan Li","year":"2022","venue":"IEEE 17th International Conference on Control & Automation (ICCA)","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhou et al., 2022] Xiaoyi Zhou, Tiange Yang, Yuanyuan Zou, Shaoyuan Li, and Hao Fang. Multiple sub-formulae cooperative control for multi-agent systems under conflict- ing signal temporal logic tasks. IEEE Transactions on In- dustrial Electronics, 2022.","title":"Multiple sub-formulae cooperative control for multi-agent systems under conflicting signal temporal logic tasks","authors":"Xiaoyi Zhou, Tiange Yang, Yuanyuan Zou, Shaoyuan Li, Hao Fang","year":"2022","venue":"IEEE Transactions on Industrial Electronics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}