Machine-generated Explanations of Engineering Models: A Compositional Modeling Approach

1. Introduction

This paper describes a method for generating explanations of how devices work. The method is implemented in the Device Modeling Environment (DME), a system to help engineers to explore the behavior of modeled devices under various sce­ narios by assisting with the formulation of simulation models and the interpretation of results [ 12, 17]. On the basis of an initial model and the behavioral predictions obtained through simulation, DME can answer a range of user queries about the structure and behavior of the modeled system.

The approach combines several techniques:

• automatically assembling mathematical models from high­ level model specifications, and explaining the low-level simulation data in terms of the original specifications

Funding was provided by NASA Grant NCC2-537 and NASA Grant NAG 2-581 (under ARPA Order 6822).

concepts such as idealized components, physical processes, and physical laws.

A method for generating explanations, then, must take into account the process by which the model was developed and simulated; it must explain the model, not just the predictions.

DME is designed to explore this idea. Model formulation, simulation, and explanation services are integrated in an in­ teractive system to help engineers build and understand models of physical devices. The explanation system in DME uses information about the derivation of the mathematical model from the original primitives, knowledge about the sim­ ulation method, and the initial conditions to answer user questions. To describe how this works, we need to look at how the system does model formulation and simulation.

2.1. Compositional Modeling in DME

DME's model formulation is based on the compositional modeling approach [4]. In compositional modeling, the mod­ el formulation system assembles a complete device model by selecting and composing modular primitives. The system starts with a library of model fragments, representing ideal­ ized components (e.g., resistors, transistors, logical gates, valves, pipes), physical processes (e.g., flows, heat transfers), operating modes of components (valves being open, transis­ tors being saturated), and other modular descriptions of physical mechanisms. In DME, model fragments may inherit properties from more general model fragments in a subsump­ tion hierarchy. For example, a generic model fragment for hydraulic components, which posits terminals for fluid flow, might be inherited by model fragments for pipes and valves. Each model fragment has a set of applicability conditions that specify when the model makes sense (e.g., a valve operating mode makes sense only for components modeled as valves). Model fragments also have activation conditions that indi­ cate when an applicable model holds in a given simulation (e.g., the model of a boiling process can only hold when the temperature of the water is above some threshold). Each model fragment contributes a partial description of the behav­ ior that occurs when the conditions hold. The behavior is specified in terms of algebraic and logical constraints on the values of simulation parameters.

The user specifies a set of abstract components and initial values for parameters in a scenario description. If the initial conditions are incomplete (i.e., the resulting model is under constrained), the system prompts the user for more con­ straints. From the scenario description, the system assembles a set of applicable model fragments and composes their par­ tial behavior descriptions into a mathematical behavior model (a system of constraint equations and rules for discrete events).

The mathematical model is then used to drive a simulation, which predicts time-varying values of para{Ileters. In DME, the simulation is performed by a distinct module, using off­ the-shelf techniques for numerical integration [2] and quali­ tative simulation [15]. During any point in the simulation, the behavior of the modeled system is defined by a set of govern­ ing equations, called the equation model, and the values of the parameters constrained by those equations. The equation model is the union and composition of the equations contributed by model fragments whose activation conditions are satisfied.

A qualitative state is a period during which a set of equa­ tions remains unchanged. Within a qualitative state, the values of continuous parameters change. When parameter values cross limit points, the activation conditions of some model fragments become true and others become false, lead­ ing to a state transition. DME continually monitors the numerical simulation for such changes. When limit points are reached, or when discrete events trigger a state change, DME updates the equation model for the new state. Thus, in DME, model formulation and simulation are performed dy­ namically and synchronously.

We will show examples of DME's operation from the do­ main of the Space Shuttle's Reaction Control System (RCS). The RCS is a system of thrusters that are used to steer the ve­ hicle. To investigate the design and testing of operator procedures, a model library for the pipes, tanks, and valves of the RCS was built in DME. Figure I shows a schematic dia­ gram of the model. The RCS model contains 160 components, 150 equations, and 180 parameters. The scenar­ io description is a description of these components, their connection topology, and the initial states of valves and tanks.

3. Explanation Generation

DME uses a variety of techniques to produce reasonable qual­ ity natural language text in an interactive setting without requiring sophisticated models of users, discourse structure, or natural language. We describe the essential elements of the design in this section.

3.1. Query-driven explanation generation

At any time during a simulation, the user can ask DME for an explanation. Each question is a query parameterized for some class of objects (a component, operating mode, qualita­ tive event, parameter, or state). For each class of questions, there is a task-specific, domain-independent procedure for generating answers called an explanation schema [18]. The explanation types currently implemented include:

• What is this component? Describes the parts, parameters, and possible behaviors (e.g., Figure 2)
• What is the definition of this parameter, relation, component, or operating mode?
• What are the initial conditions of this simulation scenario?
• What is happening in this state? Describes a snapshot of the simulation for a qualitative state
• What just happened? Summarizes salient changes in a simulation state (Figure 3)
• What caused this event? Explains the logical preconditions of a model fragment that asserts a discrete change
• What caused this change in behavior? Explains activation conditions of an operating mode (Figure 4)
• What influences this parameter? Gives causal account of the how this parameter obtains its value in a given current state (Figure 5)

To initiate an explanation, the user clicks on some graphics or text that has been presented. The system schematic is a useful starting point. A schematic like that shown in Figure I, created using DME modeling tools, is shown to the user in a window. Each component and link is mouse sensi­ tive.

When the user clicks on a presented object, the DME offers a set of relevant questions. For example, in Figure 2 the user clicks on one of the thruster icons in the picture and asks "What is this component?" The resulting explanation is gen­ erated from the static structure of the thruster model and the set of model fragments representing its possible operating modes.

3.2. Identifying salient information

By clicking on the phrase "in state 7," the user then asks for an explanation of "what just happened." The answer is shown in Figure 3. In this type of explanation, only those events that can cause qualitative state transitions are reported. In Figure 3, the system reports that the value of a quantitative parameter (pressure margin of thruster 1) crossed a limit point (0 Pascals). Limit points are derived from an ·analysis of the activation conditions of model fragments; they are values that could cause models fragments to become active or inactive. In this example, the model fragment for thruster 1 working normally became inactive, and the model fragment for thrust­ er I cavitating became active.

Limit point analysis, which is common in qualitative sim­ ulation, is used as a salience heuristic for explanation. This achieves a dramatic reduction in the information that might be displayed: of the 180 parameters in the RCS model, the "what just happened" explanation only mentioned PMrhrl· In a conventional simulator, the trajectories of some subset of the modeled parameters can be displayed. However, they would typically be chosen in advance and fixed throughout the sim­ ulation. Because DME is responsible for updating the equation model, it can dynamically determine salient events and filter the information presented.

3.3. Follow-up questions grounded in presentations

All explanations are presented using mouse-sensitive text that allows the user to ask follow-up questions. Every noun phrase, verb phrase, or sentence in an explanation represents some object, event, or fact for which there are anticipated questions and corresponding explanation schemata. For ex­ ample, in Figure 3 the strings "pressure margin" "thruster I" "PMrhrl" and the entire sentence are all mouse-sensitive re­ gions with separate menus of relevant follow-up questions. This technique of associating follow-up questions with mouse-sensitive text has been used by Moore and Swartout [ 19] to help manage an interactive dialog based on a text plan. Although DME does no text planning, the technique does al­ low the user some control over the level of detail and focus of the explanation. This lessens the need for a user or dialog model, especially when tutoring is not the explanation task.

3.4. Compositional text generation

The text presented in DME explanations is composed from textual annotations on various model fragments. In Figure 2, "Thruster 1" is an annotation on the relation between the RCS system model fragment and the thruster model. The name and definitions of "intake terminal" comes from the generic fluid terminal model, which is a component of the thruster model. The text used to describe the quantitative parameters comes from text fragments associated with the models for those pa­ rameters. For instance, "pressure threshold" is the name for the one of the parameters of a thruster; "PT" is its abbrevia­ tion, and "the minimal pressure that should be applied ... " is its definition. The texts identifying the four operating modes for a thruster come from the four model fragments represent­ ing these behaviors. The generation of even this simple summary is completely driven by the model library; model fragments and textual annotations can be added, deleted, and modified independently. This modularity is possible because the behavior and structure of the thruster are determined by composing model fragments from a library.

Noun phrases that identify subcomponents also reflect the part composition of the device; for example, in Figure 5 the phrase "the pressure at the intake terminal of thruster 1" is composed from the annotations on the pressure parameter ("pressure" "at"), the relation between the thruster and the terminal ("intake terminal"), and the relation between the RCS and thruster 1. A similar set of annotations was used to generate the associated glyph "Pin(Thr1)·"

The model fragment representation is composable because the partial descriptions of behavior represented by each frag­ ment can be composed into a meaningful global behavior. Compositionality allows libraries of reusable model frag­ ments. Similarly, the text generation technique is composable because the text fragment annotations can be composed into English sentences. Writing these annotations requires no knowledge of linguistics.

The problem with generating sentences by composing text fragments is that the compositions can be repetitive and ver­ bose. To address this problem, DME applies several transformations to composed text. · Abbreviations for compo­ nents and model parameters are introduced, and subsequent references are treated as if the abbreviations were pronouns (Pin(Thr1)). Since descriptions of device structure and behav­ ior often introduce parallel structure, factoring transformations are used to eliminate repeated references to the same component, parameter, or parameter value. Other transformations include capitalization, punctuation, indenta­ tion, and enumeration. This class of language transformation is described in [ 11].

4. Causal explanation

The explanations shown so far reflect the structure of the modeled device and state of the simulation program. They are relatively easy to generate because the necessary informa­ tion about the models and simulation history is represented explicitly in a knowledge base. More difficult are explana­ tions of the causes of events or states. DME can answer two classes of questions about causality: what causes a change in qualitative state, and what causes a quantitative parameter to have its current value.

4.1. Explaining logical preconditions of events

The first type of causal explanation is a logical account of a discrete change in value or state. Discrete changes in the op­ erating regions of components or discontinuous changes in quantitative parameters are reflected in the changing activa­ tion of model fragments. The "cause" of a discrete change, then, can be explained in terms of the predicates and parame­ ter values that are mentioned in the activation conditions of a model fragment. DME explains how the activation conditions of model fragments become satisfied (or cease to hold) in a given state or at a state transition. It can then recursively ex­ plain the causal ancestry of each of the parameter values mentioned in the conditions (on request). For instance, in Figure 4, the user asks why the thruster is now cavitating. The resulting explanation describes the conditions under which the "cavitating from low pressure" operating mode is active, and why those conditions held in state 7. The first condition is that the discrete parameter "flow status" must have the value "open." In this case, the parameter got its cur­ rent value because of an operator action in an earlier state. The second condition in Figure 4 is an inequality over the pressure margin parameter, which is a continuous quantity. Its current value is displayed to show that the inequality is sat­ isfied.

Explaining how model fragment preconditions are satis­ fied is similar to explaining a sequence of rule firings in rule­ based expert systems. In these systems, such explanations make sense only when the rules correspond to meaningful in­ ferences or actions in the system being modeled [1]. Similarly, in compositional modeling, explanations of chang­ es in qualitative state make sense if the transitions among model fragments reflect meaningful physical changes in the modeled system. In engineering domains, components are designed to operate in predicable modes (e.g., open, closed, broken), and the operating assumptions of models of physical mechanisms can be made explicit (e.g., boiling starts at a boiling point).

4.2. Explaining causal influences on a continuous parameter

While the "cause" of a discrete event or parameter can be traced to the satisfaction of logical preconditions, the "cause" of the value of a continuous quantitative parameter is more complex. In Figure 5, the user asks why the pressure margin has a negative value. The answer is an account of the mathe­ matical influences on the parameter in the current equation model.

• j~e pressure margin of thruster 1 (PM.ntr1) 1, jdropped below OPa.

As explained in Figure 5, the pressure margin of thruster I (PMThr1), is determined by the intake pressure P1n(Thr1), due to the equation that defines the pressure margin parameter (which comes from the thruster model) and the fact that the other term in that equation is an exogenous constant. The pressure at the intake terminal, in tum, is equal to the pressure at the output terminal of manifold I, which is equal to the pressure at the intake terminal of the manifold. This chain of equalities (a=b, b=c, ... ) can be followed through seven subse­ quent terminals of isolation valves and pipes to the pressure at the output terminal of the oxygen tank (P out(o2.1ank)). This would be tedious to explain to the user, so by default the sys­ tem collapses this chain into the single equation shown in the second paragraph of FigureS. In this explanation, instead of showing the chain of equations, the system displays all the ac­ tive operating modes that contributed an equation in the chain. For example, the phrase "manifold I was open" is a de­ scription of the "open" operating mode of the "manifold!" component. If the manifold were closed, a different equation would be in force. Finally, the oxygen tank pressure is shown to be determined by the quantity of gas in the tank (No2·1ank), as given by the ideal gas law associated with the tank.

The explanation goes beyond a literal presentation of the equations in several respects. First, it is causally directed; the explanation traces the path of influence back from the thruster through pipes to the oxygen tank. Second, it is an abstraction; most of the equations are hidden or summarized. It also re­ lates the mathematical relationships among parameters to the high-level model fragments that contributed the relevant equations (the third equation is an application of the ideal gas law, which came from modeling the oxygen tank as an ideal container).

4.2.1 Causal ordering

This explanation happens to follow the connection topology shown in Figure I, but to the simulator there are only con­ straint equations. In a different scenario, the direction of causality over the same components could be reversed (e.g., back pressure). Instead, DME infers the path of causal influ­ ence from oxygen tank to thruster using the causal ordering algorithm [13]. Given a set of equations constraining a set of model parameters, and the knowledge of which parameters are exogenous (i.e., determined outside the modeled system), the algorithm produces a dependency graph over the remain­ ing parameters. The graph determines the order in which parameter values are computed within a numeric simulation loop, and usually corresponds to the intuitive assignment of causal influence in the model.

4.2.2 Filtering irrelevant information

In models of even moderate size, there may be many param­ eters that are involved in the behavior of interest. However, to be comprehensible, an explanation must be limited to just a few of the most salient parameters. In explaining the influ­ ences on a continuous parameter, DME makes use of several heuristics to determine which nodes of the causal order graph are worth mentioning. These heuristics include:

• recognizing and collapsing equality chains
• looking at the physical dimensions of the parameters involved (changes in dimensions are interesting)

• looking at the position of variables in the causal ordering (root nodes are interesting)

In the example of Figure 5, of the 15 non-exogenous pa­ rameters involved in the path from PMrhrl to N02.1ank· only Pln(Thr1) and Pout(02-tank) were explained. A long chain of equalities was collapsed to the single equation Pln(Thr1) = Pout(02-tank)· The procedure for filtering parame­ ters and collapsing equality chains is not dependent on knowledge about flows or device topologies. Details of the use of causal ordering to produce explanations in DME are given in a separate paper [9].

The important consequence of inferring causal order is that it avoids the need for ad hoc "causal models" that are built specifically for explanation generation. In explaining how things work, people often use causal terminology. However, when analyzing the behavior of electromechanical devices, engineers use logical and mathematical constraints, which do not specify causality. Inferring causal dependencies allows the use of realistic engineering models without sacrificing the ability to generate causal explanations.

5. Related Work

Existing systems for generating explanations of device be­ havior typically depend on models built explicitly for the explanation or tutoring task [8,21]. When explanations are generated from more general behavior models, the explana­ tions typically follow from hard-coded labeling of causal influence [24] or component function [14]. DME model li­ braries are designed primarily for describing behavior for engineering analysis. Textual annotations used for explana­ tion are optional and have no behavioral semantics.

Integrating model formulation and explanation in DME is analogous to the approach used in the Explainable Expert System project [22]. In EES, the developer gives high-level specifications of a knowledge system, and the low-level code is synthesized. Then, the answers produced by the compiled knowledge system can be explained in terms of the original specifications. In DME, the model builder gives high-level descriptions of a device, behaviors of interest, and assumed operating environment, and the system generates a low-level equation model for simulation. It can then explain the pre­ dicted behavior in terms of the high-level descriptions.

DME's explanation approach is most similar to the ap­ proach taken in the SIMGEN systems [3,7]. Like DME, these systems use a compositional modeling method of model for­ mulation, perform numeric simulation, and generate causal explanations. However, they use a qualitative model for ex­ planation and a parallel quantitative model for simulation. DME generates qualitative explanations from the quantitative model. In SIMGEN, the qualitative model is used to produce an envisionment [5,6], which is a description of all possible qualitatively distinct states of the device. A numerical simu­ lation is run in lock-step with the qualitative model. A fundamental limitation of this approach is that envisionment requires a complete enumeration of the state space of the modeled system; in complex devices, this space is very large and impractical to enumerate.

In SIMGEN, causal explanations are produced directly from the causal influences represented explicitly the qualitative model. Thus, SIMGEN requires that the model library ("domain theory") include an integrated network of qualita­ tive and quantitative model fragments that anticipates the relevant causal dependencies. DME only requires model fragments containing the quantitative constraint equations and discrete event rules. The intent is to facilitate the devel­ opment of model libraries by domain specialists not trained in qualitative modeling. In any case, useful model libraries are difficult to build and new development tools will be required to achieve this goal.

The successor to SIMGEN, SIMGEN.MK2 [3], addresses the efficiency and scale problems with a compilation ap­ proach. SIMGEN.MK2 precomputes the conditions for state transitions from an analysis of the qualitative model, without using a global envisionment. Nevertheless, the time required by SIMGEN.MK2 to produce a self-explanatory simulator for a non-trivial example is still significant (on the order of hours). In contrast, DME uses an interpreted approach, with a different performance profile. DME dynamically reconfig­ ures the model at each qualitative state transition. Although it is fast at doing numerical simulation within a qualitative state, DME is slower at computing the set of active model fragments at the transition to each new state. However, it can generate a complex explanation of existing states almost in­ stantaneously. This facilitates the rapid configuration and debugging of models, which is important for design tasks.

6. Summary and future work

We have described a collection of techniques that, when tak­ en together, constitute a practical method for generating interactive explanations of device behavior in natural lan­ guage. The main result is the ability to explain engineering models of the sort engineers use for analysis (constraint equa­ tions with ordinary differential equations). Integrating explanation with model formulation capabilities allows DME to explain numerical values predicted from equation models in terms of salient behavioral abstractions such as physical processes, idealized components, and operating modes. Us­ ing compositional text generation facilitates modularity and scalability in the development of model libraries. Just as the behavior of a component can be described without making as­ sumptions about the connection topology in which it operates, the textual annotations on model fragments can be written without anticipating the sentences into which they will be composed. Because causal relationships are inferred, there is no need to build assumptions of causality into the models.

Our long-term objective is to make machine-generated ex­ planation part of the engineer's toolbox. High-quality explanations could provide support for conceptual design, testing operator procedures, and documenting design intent. We believe that achieving these goals requires the following new capabilities:

• The ability to generate and explain alternative, hypothetical (what-if) scenarios
• Explaining the function of a device in terms of its structure and behavior (e.g., how a functional requirement is satisfied). This will build on work in representing function [23]

• Active documentation of device behavior and intended behavior, in which stand-alone, explainable simulations with interactive, query-driven explanations are used as a medium for documentation of designs [I 0]
• Explaining modeling assumptions, such as the reasons for making approximations. This will require model formulation systems to reason explicitly about the choice of models, as explored in recent work [4, 16,20]

Work on these problems is in progre~~.

Acknowledgments

The DME system is the product of members of the How Things Work project, including Richard Fikes, Yumi Iwasa­ ki, Alon Levy, Chee Meng Low, Fritz Mueller, and James Rice. Brian Falkenhainer and Ken Forbus have been very in­ fluential and provided valuable comments on the draft.