Towards a Resilient Intelligent Automation System

Towards a Resilient Intelligent Automation System

Segev Shlomov, Sami Marreed, Avi Yaeli

Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence
Demo Track. Pages 8797-8800. https://doi.org/10.24963/ijcai.2024/1035

Intelligent Process Automation (IPA) solutions must adapt to changes in user interfaces autonomously, without manual intervention. Addressing this critical challenge and aiming to advance the state-of-the-art, last year we introduced the IPA Challenge competition at IJCAI 2023. This demo paper presents IDA - our novel UI automation solution, developed to tackle complex resiliency issues. Leveraging the capabilities of large language models and employing grounded instructions, our system demonstrates a significant advancement towards resilient IPA. We provide an overview of the IPA Challenge, detail the architecture of our system, and illustrate its effectiveness in overcoming the resiliency challenges. A link to the demo video can be found at: https://youtu.be/G5nI3V9Umjc
Keywords:
Humans and AI: HAI: Intelligent user interfaces
Agent-based and Multi-agent Systems: MAS: Trust and reputation
AI Ethics, Trust, Fairness: ETF: Safety and robustness
Humans and AI: HAI: Applications
Humans and AI: HAI: Other
Natural Language Processing: NLP: Language grounding