PeopleFinder: a Multimodal Multimedia Communications Tool for Interconnecting Office Staff

PeopleFinder: a M ultimodal Multimedia Communications Tool for Interconnecting Office Staff

Innes A. Ferguson and James D. Davlouros

Knowledge Systems Laboratory (liT) National Research Council Ottawa, ON, Canada

innes@ai.iit.nrc.ca; james@cais.dsm.sp-agency.ca

Abstract

The PeopleFinder is a knowledge-based tool to assist users in determining the whereabouts of other staff located in an office or network envi­ ronment. The tool makes use of several modes of input and output, as well as employing a num­ ber of interface and communications media with which to present information and interconnect geographically distributed system users. The ac­ companying video contains example uses of the tool which help illustrate some of its functional­ ity.

PeopleFinder

The PeopleFinder employs agent-oriented design tech­ niques as a way of integrating a variety of heteroge­ neous applications running on a number of different computing platforms and operating systems. Specifi­ cally, the PeopleFinder's functionality is carried out by a collection of coordinated software agents which can

1. assume user-delegated tasks (e.g. interpreting spo­ ken commands, invoking email and voice recording tools, dialing out on a phone);
2. perform a number of application level tasks (e.g. up­ dating the graphical user interface, applying heuris­ tics to decide whether and/or how system users want to be contacted); and
3. carry out a number of lower level operating system level activities, the majority of which will be be­ yond the level of expertise of most casual system users (e.g. translating between different speech en­ coding formats, establishing which users are logged on to the various multi-platform computer networks throughout the building).

The agents used in the PeopleFinder application are based on the CALVIN open agent framework (Fer­ guson and Davlouros 1995) which in turn is an ex­ tension of the TouringMachine architecture (Ferguson 1992). 1 This framework provides application develop­ ers with a powerful set of agent programming tools including libraries of intra- and inter-agent protocols (e.g. KQML 2 ), sensory and effectory apparatus, in­ ternal behavior APis, persistent storage management, and (currently under consideration) CO RBA compli­ ance.

1 The Communicating Agents Living Vicariously In Networks (CALVIN) architecture is an agent frame-

Besides providing an added level of flexibility and robustness to the overall system behavior (Ferguson 1995), the existence of multiple autonomous - and, therefore, concurrently operating - agents also facili­ tates the application of multiple channels of interaction between human and system (Faure and Julia 1994). A number of similar benefits and issues have been identi­ fied in the application of the Open Agent Architecture (Cohen et a/. 1994) to the management of email; in particular, the roles of such agent skills as delegation ("the ability to receive a task to be performed with­ out the user's having to state all the details"), data­ directed execution ("the ability to monitor local or re­ mote events, such as database updates, OS, or net­ work activities"), and communication ("the ability to enlist other agents ... in order to accomplish a task") . In addition, the PeopleFinder can be seen to offer a number of desirable features which are characteristic of technology for telepresence; that is, technology which provides, despite geographical or temporal distance, a sense of social proximity - a kind of social prosthesis for overcoming gaps and weaknesses with an organiza­ tional structure (Giachino 1993; Buxton 1994).

Briefly, some of the key design features of the Peo­ pleFinder include:

• Agent-oriented design. The system is based on a number of coordinated autonomous agents, each specialized in one or more system tasks, a number of which have been mentioned above.
• Multimodal interaction. The system makes use of various input and output modalities for human-

work developed at the National Research Council's Knowledge Systems Laboratory. See WWW page http:// ai.iit.nrc.ca/CALVIN /title.html for more details.

2 The Knowledge Query and Manipulation Language (KQML) is a protocol intended to support interoperability among intelligent agents in distributed applications (Finin et al. 1992; Werkman 1994)

computer interaction; in particular, the keyboard, mouse, and speech for input, and audio and screen­ based feedback for output.

• Multimedia presentation. The system makes use of a number of different media for human­ computer interaction; in particular text, graphics, animation, and prerecorded video (the latter act­ ing as a "cheap", but nevertheless very effective, substitute for in-office video cameras and/or active badges 3 )
• Communications-oriented capabilities. The system enables transparent communication across different computer platforms (Macintosh, Unix) and facilitates the interconnection of system users via telephone, email, and voice messaging.

A number of features of the CALVIN architecture have proven useful for developing multimodal appli­ cations that integrate a number of distributed media resources. In particular, rapid responses to users' com­ mands are facilitated through integration of appropri­ ate reactive behaviors in the system's Interface and User agents (Ferguson and Davlouros 1995); in addi­ tion, blending of complementary input modalities is facilitated through the execution of multiple concur­ rent agents (which in turn are able to execute multiple concurrent, task-specific behaviors).

Current work already underway includes porting the graphical user interface portion of the PeopleFinder to both PC and Unix platforms (in the interest of extend­ ing the tool's audience and ensuring a more thorough testing and empirical evaluation phase of the project); integrating a number of other software applications such as teleconferencing, voice dictation, and video camera-based face recognition; extending agents' ca­ pabilities for autonomously resolving run-time conflicts resulting from shared access to the different presenta­ tion and communications resources used by the system (see Werkman's KBN negotiation-based conflict reso­ lution work for related issues (Werkman 1994)); and formalizing the various rules used by the PeopleFinder to combine multiple media with multiple modalities for both human-computer interaction and computer sup­ ported human-human communication, much along the lines of the work of Arens et a/. (Arens et a/. 1993) on allocating multiple media.

The tool is implemented using a variety of differ­ ent scripting languages (AppleScript, Quickeys, and C­ shell) and runs on a Macintosh Quadra 840 AV. The tool also makes use of the Macintosh's Apple Phone tool and Geoport Telecom Adapter for performing its various computer-telephony integration tasks.