WIP: From Multimedia to Intellimedia

WIP: From Multimedia to Intellimedia

Elisabeth Andre, Wolfgang Finkler, Winfried Graf, Karin Harbusch, Jochen Heinsohn, Anne Kilger, Bernhard Nebel, Hans-Jiirgen Profitlich, Thomas Rist, Wolfgang Wahlster (Authors of the WIP System), Andreas Butz (Video Producer), Anthony Jameson (Speaker)

German Research Center for Artificial Intelligence (DFKI)

Stuhlsatzenhausweg 3, D-66123 Saarbriicken

Email: {name}@dfki.uni-sb.de

This video provides an overview of the WIP project . WIP aimed at the development of a presentation system that is able to generate a variety of multimedia docu­ ments considering generation parameters, such as target group, presentation objective, resource limitations, and target language (see Fig. 1) .

The major components of the WIP system are: a pre­ sentation planner that is responsible for determining the contents and selecting an appropriate medium combina­ tion, medium-specific generators and a layout manager that arranges the generated output in a document. The video gives a survey of the WIP architecture and demon­ strates the performance of the single modules by several system runs.

A basic assumption behind the WIP model is that not only the generation of text and dialog contributions, but also the design and presentation of graphics and multi­ modal documents are planning tasks. As input, WIP re­ ceives a presentation goal (e.g., the user should know how to prepare a modem for reception of data.). This goal is forwarded to the presentation planner which tries to find a presentation strategy which matches this goal and gen­ erates a refinement-style plan in the form of a directed acyclic graph (DAG) . This DAG reflects the proposi­ tional contents of the potential document parts, the in­ tentional goals behind the pints as well as the rhetorical relationships between them. While the top of the pre­ sentation plan is a more or less complex presentation goal (e.g., instructing the user in switching on a device), the lowest level is formed by specifications of elementary presentation tasks (e.g., formulating a request or depict­ ing an object). These elementary tasks are directly for­ warded to the medium-specific generators, currently for text and graphics.

The design of the text generator was strongly influ­ enced by the quest for incremental processing. The first component that is activated during natural language generation is the text design component. As soon as the presentation planner decides that a particular ele­ ment should be presented as part of a text, the element is handed over as input to this component. The main task of the text design component is the organization of input elements into clauses. This comprises the determi­ nation of the order in which the given input elements can be realized in the text and lexical choice. The results of the text designer are preverbal messages. These preverhal messages are forwarded in a piecemeal fashion to the text realization component where grammatical encoding, linearization and inflection take place. The text realiza­ tion component is based on the formalism of Lexicalized LD /LP Tree Adjoining Grammars.

As the text generator, the graphics generator includes components for design and realization. The task of the graphics design component is to transform presentation tasks received from the presentation planner into a se­ quence of operators to be executed by the graphics re­ alization component. Basic knowledge about how to ac­ complish this transformation is represented by so-called design strategies. The realization component can be con­ sidered as an extension of an object-oriented graphics editor that handles both two-dimensional concepts and three-dimensional models of objects and object configu­ rations. Thus, the component has to support operators that manipulate models, mappings and pictures. Beside these operators, the functionality of the realization com­ ponent also encompasses evaluation operators (e.g., to check whether an object as part of an object configu­ ration is visible from a given viewing specification, or to check whether a picture part can be discriminated from other picture components). These evaluators are necessary in order to recognize whether the effect of an achievement operator has been destroyed by the appli­ cation of subsequent achievement operators.

The main task of the layout manager is to convey cer­ tain semantic and pragmatic relations specified by the planner by the arrangement of graphic and text frag­ ments received from the medium-specific generators, i.e., to determine the size and the exact coordinates of the entities for positioning them on the document page. We use a grid-based approach as an ordering system for effi­ ciently designing functional (i .e., uniform, coherent and consistent) layouts. The automatic placement of layout objects in a design space can be viewed as a combina­ torical problem. Therefore, we treat layout as a con­ straint satisfaction problem in a finite discrete search space. We encode graphical design knowledge via pri­ oritized constraints expressing semantic/pragmatic and geometrical/topological relations.

WIP has been designed for interfacing with heteroge­ neous back-end systems such as expert systems, tutoring systems, intelligent control panels , on-line documenta­ tion and help systems, which supply the presentation system with the necessary input . While for each domain the application knowledge and the wireframe model are fixed, the presentation goal and the generation param­ eters can be varied. The intelligence of the WIP sys­ tem lies in its ability to present the same information in a variety of ways depending on the generation pa­ rameters. Thus WIP allows for tailoring presentations for individual users in particular communicative situa­ tions. To demonstrate this, the video contains a variety of presentations generated by the system starting from the same presentation goal, but different settings of the generation parameters.

WIP can also be used in a stand-alone fashion, where an author specifies the necessary domain information. This leads to the long-term vision of an intelligent au­ thoring system, that forces one to specify information only once in a formal way and then allows the gener­ ation of a possibly infinite variety of presentations of this information tailored to various audiences and me­ dia. In contrast to the current situation in technical writing and document preparation, this approach - sim­ ilar to the view concept in database design - could en­ sure consistency across all derived presentations, since the underlying content is stored in only one place.

The WIP project was supported by the German Min­ istry of Research and Technology under grant ITW 8901 8.