{"type":"ScholarlyArticle","title":"Are Large Language Models Fluent in Declarative Process Mining?","authors":[{"name":"Valeria Fionda","slug":"valeria-fionda","given":"Valeria","family":"Fionda","affiliation":""},{"name":"Antonio Ielo","slug":"antonio-ielo","given":"Antonio","family":"Ielo","affiliation":""},{"name":"Francesco Ricca","slug":"francesco-ricca","given":"Francesco","family":"Ricca","affiliation":""}],"abstract":"Recent advancements in AI have made LLMs valuable tools for automating the interpretation of textual descriptions of business processes and for converting formal process specifications into natural language. However, there are no practical methodologies or systematic assessments to ensure these automatic translations are faithful. This paper proposes a novel approach, based on an auxiliary bidirectional translation task, to assess LLMs performance quantitatively; also, it also empirically evaluates the performance of state-of-the-art LLMs for bidirectional translations between natural language and declarative formal process specifications. The results reveal substantial variability in performance among the LLMs, highlighting the importance of LLM selection and confirming the need for a robust method for assessing LLMs' outputs.","doi":"10.24963/ijcai.2025/501","url":"https://ijcai.org/proceedings/2025/501","pdf":"https://ijcai.org/proceedings/2025/0501.pdf","markdown":"https://ijcai.org/proceedings/2025/501.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/501","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Knowledge Representation and Reasoning","track":"Main Track","number":501,"pages":{"first":4500,"last":4508},"keywords":["Knowledge Representation and Reasoning: KRR: Qualitative, geometric, spatial, and temporal reasoning","Knowledge Representation and Reasoning: KRR: Applications"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}