{"type":"ScholarlyArticle","title":"Improving Consistency Identification in Task-oriented Dialogue Through Multi-Agent Collaboration","authors":[{"name":"Peng Wang","slug":"peng-wang","given":"Peng","family":"Wang","affiliation":""},{"name":"Shuo Li","slug":"shuo-li","given":"Shuo","family":"Li","affiliation":""},{"name":"Ruoxi Zhou","slug":"ruoxi-zhou","given":"Ruoxi","family":"Zhou","affiliation":""},{"name":"Qiguang Chen","slug":"qiguang-chen","given":"Qiguang","family":"Chen","affiliation":""},{"name":"Xiao Xu","slug":"xiao-xu","given":"Xiao","family":"Xu","affiliation":""},{"name":"Hao Fei","slug":"hao-fei","given":"Hao","family":"Fei","affiliation":""},{"name":"Dagang Li","slug":"dagang-li","given":"Dagang","family":"Li","affiliation":""},{"name":"Wanxiang Che","slug":"wanxiang-che","given":"Wanxiang","family":"Che","affiliation":""},{"name":"Libo Qin","slug":"libo-qin","given":"Libo","family":"Qin","affiliation":""}],"abstract":"Consistency identification in task-oriented dialog (CI-ToD) typically consists of three sub-tasks: User Query Inconsistency (QI) identification, Dialogue History Inconsistency (HI) identification, and Knowledge Base Inconsistency (KBI) identification, which aim to determine inconsistent relationships between system response and user query, dialogue history, and knowledge base. Previous approaches focus on the exploration of deep learning models for CI-ToD. While these models achieve remarkable progress, they still rely on large amounts of labeled data, which is hard to achieve in real-world scenarios. Motivated by this, in the paper, we aim to explore large language models for CI-ToD, which do not require any training data. In addition, we further introduce a multi-agent collaboration framework (MAC-CIToD) to model the interaction across three sub-tasks in CI-ToD, including (1) Full Connection paradigm, (2) Cycle Connection paradigm, and (3) Central Connection paradigm, which effectively builds interaction across QI, HI, and KBI. Experiments on the standard benchmark reveal that our framework achieves superior performance. Additionally, we compare MAC-CIToD with the most advanced trained approaches and find that its zero-shot performance on most metrics even surpasses that of models after training on the CI-ToD dataset.","doi":"10.24963/ijcai.2025/918","url":"https://ijcai.org/proceedings/2025/918","pdf":"https://ijcai.org/proceedings/2025/0918.pdf","markdown":"https://ijcai.org/proceedings/2025/918.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/918","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Natural Language Processing","track":"Main Track","number":918,"pages":{"first":8259,"last":8267},"keywords":["Natural Language Processing: NLP: Dialogue and interactive systems"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}