Abstract
Intelligent Process Automation (IPA) solutions must adapt to changes in user interfaces autonomously, without manual intervention. Addressing this critical challenge and aiming to advance the state-of-the-art, last year we introduced the IPA Challenge competition at IJCAI 2023. This demo paper presents IDA - our novel UI automation solution, developed to tackle complex resiliency issues. Leveraging the capabilities of large language models and employing grounded instructions, our system demonstrates a significant advancement towards resilient IPA. We provide an overview of the IPA Challenge, detail the architecture of our system, and illustrate its effectiveness in overcoming the resiliency challenges. A link to the demo video can be found at: https://youtu.be/G5nI3V9Umjc
Keywords
- Humans and AI: HAI: Intelligent user interfaces
- Agent-based and Multi-agent Systems: MAS: Trust and reputation
- AI Ethics, Trust, Fairness: ETF: Safety and robustness
- Humans and AI: HAI: Applications
- Humans and AI: HAI: Other
- Natural Language Processing: NLP: Language grounding
