Designing Comprehensible Agents

Abstract

For many applications, it is important that agents be not only correct, but also comprehensible to human users. Typically, people have tried to make agents' behavior and reasoning understandable by adding post-hoc specialpurpose explanation systems, with often disappointing results. Here, I instead take the comprehensibility of agent behavior as a central agent design consideration from the start. I describe an agent architecture, the Expressivator, that supports comprehensibility on top of a behavior-based framework , using four technical innovations: (1) structuring the agent's behavior according to the signs and signifiers it is intended to communicate; (2) allowing the agent to keep track of its impression on the user with sign management, (3) using behavioral transitions to explain the reasons for agent behavior, and (4) expressing behavioral interrelationships directly using meta-level controls.