LLARS: Enabling Domain Expert & Developer Collaboration for LLM Prompting, Generation and Evaluation

Steigerwald, Philipp; Stieler, Mara; Burghardt, Jennifer; Rudolph, Eric; Albrecht, Jens

doi:10.24963/ijcai.2026/995

LLARS: Enabling Domain Expert & Developer Collaboration for LLM Prompting, Generation and Evaluation

Philipp Steigerwald, Mara Stieler, Jennifer Burghardt, Eric Rudolph, Jens Albrecht

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Demo Track. Pages 8526-8529.

https://doi.org/10.24963/ijcai.2026/995

Keywords

AI: Natural Language Processing
AI: Humans and AI
AI: Multidisciplinary Topics and Applications

Abstract

We demonstrate LLARS, an open-source platform that bridges the gap between domain experts and developers for building LLM-based systems.

LLARS integrates three tightly connected modules into an end-to-end pipeline:

Collaborative Prompt Engineering for real-time co-authoring with version control and instant LLM testing,

Batch Generation for configurable output production across user-selected prompts x models x data with cost control and

Hybrid Evaluation where human and LLM evaluators jointly assess outputs through diverse assessment methods, with live agreement metrics and provenance analysis to identify the best model-prompt combination for a given use case.

New prompts and models are automatically available for batch generation and completed batches can be turned into evaluation scenarios with a single click.

Interviews with six domain experts and three developers in online counselling confirmed that LLARS feels intuitive, saves considerable time by keeping everything in one place and makes interdisciplinary collaboration seamless.

Source code: github.com/th-nuernberg/llars

Philipp Steigerwald, Mara Stieler, Jennifer Burghardt, Eric Rudolph, and Jens Albrecht. 2026. LLARS: Enabling Domain Expert & Developer Collaboration for LLM Prompting, Generation and Evaluation. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26. IJCAI, 8526–8529. https://doi.org/10.24963/ijcai.2026/995

@inproceedings{ijcai2026p995,
  title     = {LLARS: Enabling Domain Expert & Developer Collaboration for LLM Prompting, Generation and Evaluation},
  author    = {Steigerwald, Philipp and Stieler, Mara and Burghardt, Jennifer and Rudolph, Eric and Albrecht, Jens},
  booktitle = {Proceedings of the Thirty-Fifth International Joint Conference on
               Artificial Intelligence, {IJCAI-26}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Diego Calvanese},
  pages     = {8526--8529},
  year      = {2026},
  month     = {8},
  note      = {Demo Track},
  doi       = {10.24963/ijcai.2026/995},
  url       = {https://doi.org/10.24963/ijcai.2026/995},
}

TY  - CPAPER
TI  - LLARS: Enabling Domain Expert & Developer Collaboration for LLM Prompting, Generation and Evaluation
AU  - Steigerwald, Philipp
AU  - Stieler, Mara
AU  - Burghardt, Jennifer
AU  - Rudolph, Eric
AU  - Albrecht, Jens
T2  - Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2026
SP  - 8526
EP  - 8529
DO  - 10.24963/ijcai.2026/995
UR  - https://doi.org/10.24963/ijcai.2026/995
ER  -