{"type":"ScholarlyArticle","title":"LTL-Constrained Steady-State Policy Synthesis","authors":[{"name":"Jan Křetínský","slug":"jan-kretinsky","given":"Jan","family":"Křetínský","affiliation":""}],"abstract":"Decision-making policies for agents are often synthesized with the constraint that a  formal specification of behaviour is satisfied. Here we focus on infinite-horizon properties. On the one hand,  Linear Temporal Logic (LTL) is a popular example of a formalism for qualitative specifications. On the other hand, Steady-State Policy Synthesis (SSPS) has recently received considerable attention as it provides a more quantitative and more behavioural perspective on specifications, in terms of the frequency with which states are visited. Finally, rewards provide a classic framework for quantitative properties.\r\nIn this paper, we study Markov decision processes (MDP) with the specification combining all these three types. The derived policy maximizes the reward among all policies ensuring the LTL specification with the given probability and adhering to the steady-state constraints. To this end, we provide a unified solution reducing the multi-type specification to a multi-dimensional long-run average reward. This is enabled by Limit-Deterministic Büchi Automata (LDBA), recently studied in the context of LTL model checking on MDP, and allows for an elegant solution through a simple linear programme. The algorithm also extends to the general omega-regular properties and runs in time polynomial in the sizes of the MDP as well as the LDBA.","doi":"10.24963/ijcai.2021/565","url":"https://ijcai.org/proceedings/2021/565","pdf":"https://ijcai.org/proceedings/2021/0565.pdf","markdown":"https://ijcai.org/proceedings/2021/565.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/565","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Planning and Scheduling","track":"Main Track","number":565,"pages":{"first":4104,"last":4111},"keywords":["Uncertainty in AI: Markov Decision Processes","Agent-based and Multi-agent Systems: Formal Verification, Validation and Synthesis","Planning and Scheduling: Markov Decisions Processes"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}