Are Large Language Models Fluent in Declarative Process Mining?

Fionda, Valeria; Ielo, Antonio; Ricca, Francesco

doi:10.24963/ijcai.2025/501

Are Large Language Models Fluent in Declarative Process Mining?

Valeria Fionda, Antonio Ielo, Francesco Ricca

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 4500-4508.

https://doi.org/10.24963/ijcai.2025/501

Keywords

Knowledge Representation and Reasoning: KRR: Qualitative, geometric, spatial, and temporal reasoning
Knowledge Representation and Reasoning: KRR: Applications

Abstract

Recent advancements in AI have made LLMs valuable tools for automating the interpretation of textual descriptions of business processes and for converting formal process specifications into natural language. However, there are no practical methodologies or systematic assessments to ensure these automatic translations are faithful. This paper proposes a novel approach, based on an auxiliary bidirectional translation task, to assess LLMs performance quantitatively; also, it also empirically evaluates the performance of state-of-the-art LLMs for bidirectional translations between natural language and declarative formal process specifications. The results reveal substantial variability in performance among the LLMs, highlighting the importance of LLM selection and confirming the need for a robust method for assessing LLMs' outputs.

Valeria Fionda, Antonio Ielo, and Francesco Ricca. 2025. Are Large Language Models Fluent in Declarative Process Mining?. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. IJCAI, 4500–4508. https://doi.org/10.24963/ijcai.2025/501

@inproceedings{ijcai2025p501,
  title     = {Are Large Language Models Fluent in Declarative Process Mining?},
  author    = {Fionda, Valeria and Ielo, Antonio and Ricca, Francesco},
  booktitle = {Proceedings of the Thirty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-25}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {James Kwok},
  pages     = {4500--4508},
  year      = {2025},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2025/501},
  url       = {https://doi.org/10.24963/ijcai.2025/501},
}

TY  - CPAPER
TI  - Are Large Language Models Fluent in Declarative Process Mining?
AU  - Fionda, Valeria
AU  - Ielo, Antonio
AU  - Ricca, Francesco
T2  - Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2025
SP  - 4500
EP  - 4508
DO  - 10.24963/ijcai.2025/501
UR  - https://doi.org/10.24963/ijcai.2025/501
ER  -