Proceedings

Generalizing Bayesian Human-AI Collaboration: Theory and Application in Data-Scarce Environments

Peng Liu, Hailong Sun, Chung-Piaw Teo, Mabel Chou

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Main Track. Pages 3648-3655.

https://doi.org/10.24963/ijcai.2026/406

Keywords

  • AI: Humans and AI
  • Humans and AI: Human-AI collaboration
  • Humans and AI: Human-computer interaction
  • Agent-based and Multi-agent Systems: Human-agent interaction

Abstract

Combining predictions from heterogeneous classifiers—such as in-house deep learning models, human experts, and large language models (LLMs)—is a key challenge, especially in data-scarce environments such as humanitarian operations. We propose a flexible Bayesian framework to effectively fuse these diverse inputs. By integrating classifier logits with auxiliary human feedback (e.g., confidence, image clarity) using an ordered probit process, our model generalizes prior work to accommodate the real-world properties of these classifiers. We validate our framework on a challenging product recognition task in food bank operations, an environment defined by data scarcity and an inexperienced volunteer workforce. Our combined model significantly outperforms standalone ResNet, human, and LLM-based approaches, demonstrating the practical benefits of fusing these heterogeneous signal sources.

Peng Liu, Hailong Sun, Chung-Piaw Teo, and Mabel Chou. 2026. Generalizing Bayesian Human-AI Collaboration: Theory and Application in Data-Scarce Environments. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26. IJCAI, 3648–3655. https://doi.org/10.24963/ijcai.2026/406
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards