Ubiquitous Talker: Spoken Language Interaction with Real World Objects

Abstract

Augment ed reality is a research area that tri es to embody an electronic information space within the real world, through computational devices. A crucial issue within this area, is the recognition of real world objects or situations. In natural language processing, it is much easier to det ermine int erp retations of utt era nces , even if they are ill-formed , when th e co ntext or situation is fix ed . We therefore introduce robust , natural language pro cessing into a system of augmented reality with situation a war eness. Based on this idea, we have developed a portable system, called th e Ubiquitous Talk e r. This consists of an LCD display that re fl ects the scene at which a user is looking as if it is a transparent glass,· a CCD camera for recognizing real world objects with co lor-bar ID codes, a microphone for recognizing a human voice and a speaker which outputs a synthesized voice. The Ubiquitous Talker provides its user with some information related to a recognized object, by using the display and voice. It also accepts requests or questions as voice inputs. The user feels as if he/she is talking with th e object itself through the system.