What's in a Deep Model? A Characterization of Knowledge Depth in Intelligent Safety Systems

1. Introduction

Several authors have noted the distinction between 'deep' and 'shallow' models of expertise in expert systems (e.g., Hart 1982 Chandrasekaran 1982, Fink 1985). By shallow models we usually mean that conclusions are drawn directly from observed facts that characterize a situation. An advantage of shallow models is that they directly encode the heuristics that experts use in performing their reasoning tasks, and are thus relatively easy to build. rn addition, shallow models tend to be relatively efficient because they select rather than construct their sorutions. One disadvantage of shallow models, however, is that explicitly stating all the preconditions under which a solution should be selected is an error prone process. Another weakness of shallow models is that they are inflexible, unable to deal with circumstances even slightly different from ~~se explicitly anticipated (de Kleer & Brown 1984). In add1t1on, shallow models may be difficult to maintain, since what is conceptually a sfnQie piece of knowledge may be unsystematically distnbuted across several objects in a knowledge base. Finally, explanations generated from shallow models tend to be limited to traces of the chains of inference that lead to conclusions.

In contrast, deep models of expertise correspond more closely to the notion of reasoning from first principles. They tend to be more robust than shalfow models, handling problems not explicitly anticipated and exhibiting higher penormance at the periphery of their knowledge. In addition, it can be easier to verify the completeness of deep models. For example, in device-centered models of physical systems (e.g., de Kleer & Brown 1984, Davis 1984) each phys1cal device maps directly into a structured object in the representation. Deep models of expertise are also more useful for generating explanations in that reasoning steps which are usually implicit in shallow models can be elucidated. Deep reasoning is, however, bound to be slower and more complex than shalfow reasoning in that a more sophisticated contror structure is required (Koton 1985).

Abstractly characterizing deep and shallow models contrast their relative merits in a general way provi~es little directi~~ .tor knowledge engineering. ln particular, the f1eld lacks a definition of exactly whal makes a model 'deep' and lacks guidelines

· fh1s research was supported 1n part by a grants from the Army Research Office and the NASA Graduate Student Researchers program.

WHAT'S IN A DEEP MODEL?

A Characterization of Knowledge Depth In Intelligent Safety Systems ·

David Klein and Tim Flnln Computer and Information Science University of Pennsylvania Philadelphia, PA 19104

regarding the appropriate depth of models for a given application. We believe that such guidelines should be developed by abstracting from a large set of examples. In particular we advocate the approach ol (i) adopting an Informal definition' of 'knowled9e d~pth' ~ (ii) . isolating hig.h-level reasoning tasks (e.g., d1agnos1s, s1mulat1on) for analys1s, and (iii) for each such reasoning task, contrasting the merits of models inspired by various domains which vary in their relative depth.

In this ~r we sketch such an analysis for a reasonin~ task called provide a safety function' (which is defined in sect1on 3) in order to achieve two objectives: (/) To illustrate a domain­ independent mode of analysis for examining progressively deeper models, and (ii) To make the analysis available to those interested in building systems which provide safety functions. The paper is organized as follows. Section 2 provicfes a simple operational definition of 'depth' that is used in the ensuing analysis. In section 3 we define the reasoning task 'provide a safety function' and identity two subtasks which present opportunities for varying cfepth of reasoning, which are analyzed in sections 4 ana 5. Our conclusions are presented in SectiOn 6.

2. An operational definition of knowledge depth

We need a simple relation that will distinguish the depth of models of expert1se tor a given reasoning task. Our focus is on the explicit representation of knowledge 1n models of expertise, although other notions of depth whiCh highlight, for example, multipfe perspectives on a domain (e.g., Davis 1984) or notions of causality (e.g., Rieger 1977) are also potentially valuable for this purpose. The following IS intended only as an informal, operational definition of the deeper-than relation which leaves terms such as 'knowledge' and 'model' to intuition.

Definition: Consider two models of expertise M and M'. We will say that M' Is deeper-than M if there exists some implicit knowledge in M which is explicitly represented or computed in M'.

The deeper-than relation is defined over an infinite space of models of expertise for a given reasoning task. In cases where a reasoning task is decomposed into isolated subtasks which present opportunities for varying depth, the relation is applied to subtasks rather than to the composite task. For example, consider a task t which may be naturally decomposed mto subtasks t1 and t2. We address the relative depth of models for these subtasks rather than for t, tor if we build a model X and a deeper model X' for t1, and build a model Y and a deeper model Y' tor t2, then the composite models for t consisting of {X,Y'} and {X',Y} are not strictly ordered by d!=!eper-than. This occurs in intelligent safety systems as described in the next section.

3. Intelligent safety systems and knowledge depth

There has recently been great interest in intelli9ent systems that represent and reason about physical dev1ces (Bobrow 1985). One line of research concern.s the development of facilities which provide advice or take d1rect action in response to system disturbances (e.g., Underwood 1982, Nelson 1982, Ennis 1986). Of these, we focus on sys~ems which proV!de safety functions in physical sv.stem.s. Providing a sa!ety f~nct1on involves executing plans wh1ch Circumvent potential cnses in physical system environments. ·

In nuclear power operations, this encompasses executing 'a group of actions that prevent melting of the reactor core or min.imize radiation releases to the public' (Corcoran 1981). While the term 'safety function' originated in the context of nuclear facility management, we can Identify applications of the same idea in other domains, including preventing a chemical reactor in a process plant from catching fire, and preventing the depletion of o~rating system queue space in a large computer installation. Providing safety functions in such process environments may be considered an expert-level task and we will refer to systems which employ models of expertise for providing safety functions as intelligent safety systems (ISS).

Generally speaking, an ISS receives a description of the state of th~ system being controlled (the target system) as input and prOVIdes a rlan Of action for Circumventing a crisis as output. ~he wo~ o an ISS may be naturally decomposed as follows: (i) monitor target system state vanables to detect potential crisis conditions and to ascertain the status of plant component~. (ii) determine (possibly several) alternative plans for prevent1ng a crisis (plan determination), (iii) evaluate these alternatives to select tlie best one (plan evaluation) and (iv) execute or display the chosen plan. '

Our first task in examinin~J the relative merits of models which vary in depth for a g1ven reasoning task is to identify opportunities for val)'ing depth (in the sense of section 2) that provide some potential advantages (in the sense of section 1 ). For ISS's, monitoring and execution are relatively straightforward operations, but plan determination and plan evaluation may be accomplished ln a number of ways that vary in thei.r relative dep~h of reasoning. The next step in the analysis involves defining and evaluating progressively deeper models for performing each of these subtasks.

4. Reasoning depth In plan determination

We examine four progressively deeper models that may be employed to determine plans to prevent a crisis: invoking hard­ coded plans, determining plans based on hard-coded paths of components, generating plans based on system structure, and gener~tlng plans based on system structure and component behaVIOr.

4.1. Invoking hard-coded plans

The shallowest model of expertise we consider involves hard­ codinQ plans for preventing a potential crisis under various conditions. This model rs conveniently implemented In formalisms that encode situation-action pairs such as production rules. As an example, consider JESQ (Klein 1985), one of several rule-based systems that comprise YES/MVS (Ennis 1986), an expert system for managing large computer installations. JESQ's task IS to maintain a 'comfortable' level of unused space on an operating system queue, i.e., to provide the safety function 'erevent queue space depletion'. The antecedents of JESQ s rules describe the states under which the hard-coded plans in their consequents should be performed. For example, the rule in Figure 1 encodes the plan to enable a printer to print large jobs so that queue space may be freed. In effect, it enables a path of data flow from the queue to the printer.

Allow-Larqe-Joba: Priority · 5 xr ~ininq-queue-apace - low printer-aettinq · only-amall-joba amall-joba-left · few laz:qe-joba-waitinq THEN submit command to allow large joba

Figure 1: A JESQ rule

These plans, which specify the movement of data from the queue to other components (e.g., tape drives, printers), are based on the structure of the underlying computer system being modelled, but this structure Is only Implicitly represented in JESQ. As such, JESO suffers from some of the disadvantages of shallow models. For example, the conf~guratlon of the computer system may be changed requiiing modifications to this and other rules, but there is no

I

systematic way of identltvlng such modifications. In addition, JESQ's rules may omit reference to conditions in the computer system which do not usually occur, but which occasionally render encoded plans unsuccessful. Another limitation of the system is that explanations can offer little more than a presentation of the conditions under which plans are applicable. Finally, JESQ can handle only precisely those state conditions that have been anticipated.

According to our definition, a deeper model would explicitly represent the potential paths of data flow, although deeper does not necessarily imply better. For example, if configuration changes are unlikely or if the encoded preconditions are appropriate most of the time and are of manageable volume, the benefits of a deeper model of plan determination expertise might not justify the cost of its construction or the overhead of its execution.

4.2. Determining plans from hard-coded paths

The next model to be considered involves explicitly representing sets of components which may be employed to provide a safety function. This approach Is taken in REACTOR (Nelson 1982), which provides safety functions In a nuclear reactor facility. 1

Potential paths of components that can be used to cool the reactor core are encoded in a response tree as shown In Figure 2. Each path in the tree contains an instance of each of the functional components required to provide the safety function (e.g, water source, heat sink). Part of REACTOR's mission Is to select, in real time, a path composed of components that are correctly operating.

The primary advantage of this approach is its robustness. Not all potential combinations of component failures need be expncitly anticipated, since these are coordinated by the response tree structure and associated logic. Another advantage is that a change to the configuration is more easily maP,Pedlnto the representation, since paths of components are explicitly represented. Still lmpnclt, however, is the configuration description from which response trees are constructed. Since the response tree Is hard-coded, only those potential paths of components explicitly Identified in advance are candidates for selection, and configuration changes require that the resulting new paths be identified by a Kllowledge engineer. A stilr deeper model would reason directly from a scnematic to generate the potential paths. However, for applications In which the number of potential paths is manageable and the structu. re of the target system Is relatively stable, we might not be inclined to consider a deeper model.

4.3. Generating plans from system structure

The next model that we consider involves explicitly representing the configuration of the target system and using this description to generate plans for circumventing a potential crisis. As an example, we again refer to JESQ's domain (section 4.1) as cast in the structural representation of Figure 3. Using this model, plan determination adopts the form of searching a graph in which nodes represent components and edges represent their interconnections. The search always begins at node QUEUE and terminates at node USER, and each such path through the graph represents a candidate path through wh1ch datasets may flow in order to clear the QUEUE.

This representation has the advantage that a change in the configuration may be directly mapped into a change in the representation, and the knowledge about computing plans for moving datasets (searching the graph) remains ~;~nchange~ .. It is also more portable than the other representations, reqUJnng only a confi9uration description for any particular installation. This model IS deeper than the rules of section 4.1 which only implicitly represent the queue-clearing paths. This model is also deeper than the response trees of section 4.2 because we have hard-coded the mechanism by which paths are generated rather than the paths themselves.

The queue space domain permits the convenience of uniformly treating each represented device in the system, because each device is capable of accepting and stonng data. In domains that encompass devices whicfl do not exflibit this behavioral homogeneity, however, we would require a still deeper model in order to generate plans. Specifically, we would !'leed to explicitly reason about the behavior of components, s1nce not all components would play the same functional role in providing a safety function. A similar point is made by Ginsberg (1984) regarding diagnostic systems.

4.4. Generating plans from system structure and behavior

The deepest model of plan determination we consider is based on an explicit representation of target system structure and component behavior. This approach is taken in several systems that perform other reasoning tasks such as simulation (de Kleer & Brown 1984), troubleshooting (Davis 1984), and verification (Barrow 1984 ).

As an example of reasoning from structure and behavior to provide a safety function, consider the followinQ hypothetical ISS. Using a spatial representation as 1n Figure 4 (Stephanopoulous 1984) and descriptions of component behaviors, the ISS generates alternative plans for keepmg the reactor (RXR 1) from catching fire when components fail. For example, if the valve (V3) that regulates the coolant flowing to the reactor jacket becomes stuck closed, the ISS searches for a compensating action to lower the temperature of the reactor. The search proceeds both forward and backward from the reactor to yield the following alternative plans: close V4, close V5, request A to stop feed stream, request A to lower feed stream temperature, request cool temperature material from the heating system, and request the heating system to stop. For example, the ISS identifies the action 'close V4' as follows. The feedstream input to RXR1 must be such that the temperature of RXR1 is normal. Since V3 is stuck closed, the ISS must reduce either the temperature or the flow rate of the feedstream. Searching backward from RXR1, the ISS examines the behavioral model of valve V4, noting that in state CLOSED the flow rate from the valve is zero. The lSS searches a list of potential actions to find that action 'close valve' causes V4 to enter state CLOSED. Thus, the plan 'close V4' is a candidate for execution to be assessed by the plan evaluation process.

These components (heat exchangers, reactor, valves) are not behaviorally homogeneous, so reasoning about how to maintain a safe temperature in the reactor must encompass consideration of the individual behaviors of the components that impact it, as well as their position in the system structure. This model is deeper than the solely structural model of 4.3 In that the behavior of components is explicitly represented and reasoned about.

5. Varying depth of reasoning In plan evaluation

Given a set of alt~rnative plan~ for providing a _give~ safety function In a particular s1tuat1on, plan evaluation Involves selecting the 'best' one. We discuss three slgnific;ant levels of depth for evaluating alternative plans: hard-coded evaluation (explicit priorities), evaluation using utility theory and hard­ coded decision attributes (computation of priorities), and evaluation using utility theory and decision attributes which are themselves computed from a structural and behavioral model of the target system (computation of priorities and of underlying attributes).

5.1. Evaluations encoded as priorities

The shallowest representation of plan evaluation that we consider is the hard-coded numeric priority. This is a commonly employed approach; For example, a prio~ty is associated with each rule in JESQ and with each path 1n the response tree in REACTOR to provide for choosing the best when more than one are applicable.

In rule-based systems like JESQ, priorities specify relative preferences to conflict resolution. The antecedent of a rule determines the eligibility of the plan In Its conseguent and the rule's associated prionty indicates its deslrabiltty relative to other plans. Note that all objectives underlying the desirability of a plan (e.g., minimizing cost, maximizing convenience, maximizin9 the satisfaction of system users, maximizing speed) are impliCitly represented in the priority, and this gives nse to First, as the rule base grows it becomes difficult to predict the consequences of adding new rules to the knowledge base. In effect, the knowledge engineer must understand the basis for the priorities of all existing rules in order to assi1:1n a new priority to a new rule. Second, the meaning of a prionty is completely opaque, so there exists no basis for jl.lstifying a priority in an explanation. Finally, a system that uses this shallow model of choice will lack robustness. Because great importance is placed on a single heuristically justified symbol, the result of changing the priority of a single rule can significantly alter the overall behavior of an ISS.

This model of plan evaluation suffices when the number of plans is small and the relative desirability of each plan is obvious. For applications not sharing these characteristics, deeper models which explicitly represent the factors underlying priority selection may be useful for automatically justifying a choice amonQ competing plans in an explanation and for facilitating the ~ncrementar modification of the knowledge base.

5.2. Utility theory with encoded objective values

The next model to be examined involves explicitly representing the attributes and objectives underlying priority selection in the framework of utility theory. Multiattribute utility theory (Keeney & Raiffa 1976, Hansen 1983) is of particular interest in the domains of intelli1:1ent safety systems, where multiple, often mutually competitive objectives drive choices between compe11n9 plans. Under th1s approach, we regard each plan's charactenstics with respect to each of the objectives which underlie plan evaluation as arguments to a utility function which computes the priority for eacn plan. The utility function itself abstractly captures the relationships between objectives which underlie the choice of plan, and can be thought of as encoding the plant's operational policy.

Employing an explicit model of choice based on utility theory has several advantages over implicit models such as hard­ coded priorities. First, since the underlying objectives of plan evaluation are explicitly represented, a basis is provided for generating explanations regarding choices among competing plans. In addition, adding plans to the knowledge base is simplified in that the knowledge engineer need only score a new plan with respect to the aefined objectives. Where such scores are reasonably easy to formulate, this second level of depth suffices for plan evaluation. However, in applications where scores for objectives encompass consideration of the behaviors of large sets of components, formulating scores may be difficult, encouraging a greater level of depth. Specifically, we may wish to reason about (or compute) the scores for objectives rather than assign them. In applications which employ a model of plan determination which actually generates plans (e.g., those of sections 4.3 and 4.4), a deeper model of plan evaluation will be required, for there will be no way to assign scores in advance to plans which are constructed during problem solving.

5.3. Utility theory with computed objective values

The deepest model of plan evaluation that we consider for ISS's involves computing the scores for underlying objectives that are input to the utility function. This may be accomplished by formulating another set of (hard-coded) data from which objective values may be computed for each plan. In applications which employ a model of plan determination which actually generates plans, we would use the same representation of the target system to support both plan determination and evaluation. For example, consider augmenting the spatial description of the computer system of Figure 3 w1th component descriptions (e.g., processin9 time per line of data) pertaining to the objectives mentioned 1n section 5.1 (e.g., maximize user satisfactiOn). A plan (path of devices) can be evaluated with respect to turnaround t1me (one aspect of user satisfaction) by summing the processing t1mes of the processors that lie along the generated path. Other objectives (e.g., speed of action, work for the operator) would be similarly computed.

This model of evaluation is deeper than that of 5.2 because rather than encoding the values for objectives, we encode functions for computing them. One advantage of this method is that we need only supply local device-dependent data for each represented dev1ce in order to compute the desirability of actions, rather than making subjective judgements about the desirability of predefined paths. ·.

6. Summary and conclusions

We have characterized the depth of models of expertise in terms of the knowledge they explicitly represent and reason about. For the reasoning task 'provide a safe~ function', we identified two subtasks which provide opportunities for building pr~ressively deeper models of knowledge, described some part1cular models for performing each suotask, and reviewed their relative merits for some particular domains.

If knowledge engineering Is to become more of a discipline than an art, we will need to develop some guidelines which more precisely characterize 'depth of knowledge' and its implications for Intelligent system construction, performance, and maintenance. Ultimately, the guidelines would provide a basis for selecting among models of varying depth based on general domain characleristics. We believe that such guidelines should be developed by circumscribing isolated reasoning tasks and analyzing the relative merits of models of varying aepth Inspired by various domains, and we have sketched one such analysis In this paper. Future work will involve employing alternative definitions of depth In the analysis and analyzing other reasoning tasks to prov1de the data upon which the mentioned guidelines may be based.

7. References

1. Barrow, H., 'VERIFY: A Program for Proving Correctness of Digital Hardware Designs', Artificiallnlelligence 24, 1984.
2. Bobrow, D., ed., Qualitative Reasoning About Physical Systems, Elsevier, 1985.
3. Chandrasekaran, B., Mitta1 S., 'Deep verses Compiled Knowledge ~roaches to Diagnostic t<roblem Solving', Int. J. Man-Machme Studies, 1983.
4. Corcoran, W., Finnicum, D:.L Hubbard ltl1 R. , Musick, q. 1 Walzer, P., 'Nuclear Power-Plant Safety r-unctions', l'iuclear Safety vol 22, No 2, 1981.
5. Davis, R., 'Diagnostic Reasoning Based on Structure and Behavior', Artificlalliltelligence 24, 1984.
6. de Kleer, J., Brown, J. S., 'A Qualitative Physics Based on Confluences', Artificial Intelligence 24, 1984.
7. Ennis, R., Grlesmer, J., Hong, S., Kamaugh, M., Kastner, J., Klein, D., Milliken, K., Schor, M., Van Woerkom, H:~..'A Continuous Realtime Expert System for Computer Operations', I1:1M Journal of Research and Development, January 1986.
8. Fink, P., 'Control and lntagration of Diverse Knowledge In a Diagnostic Expert System', Proc. IJCAI, 1985.
9. Ginsberg, A., 'Localization Problems and Expert Systems', CBM­ TR-139, Rutgers University, 1984.
10. Hansen, P., ed., Essays and Surveys on Multiple Criteria Decision Making, Springer-Verlag, 1983.
11. Hart, P., 'Directions for AI In the Eighties', SIGART Newsletter No. 79, January 1982. ,
12. Keeney, ~"-~ Raiff~ H., Decisions with Multiple Objectives: Preferences ano Value tradeoffs, Wiley, 1976.
13. Klein, D., 'An Expert Systems Approach to Realtime, Active Management of a Target Resource', MS-CIS-85-40, University of Pennsylvania, 1985.
14. Koton, P., 'EI!tlirlcal and Model-Based Reasoning In Expert Systems', Proc. IJCAI, 1985.
15. Nelson, W., 'REACTOR: An Expert System for Diagnosis and Treatment of Nuclear Reactor Accidents', Proc. AAAI, 1982.
16. Rieger 1 C., Grlnberg 1 M., 'The Declarative Representation and ProcedUra Simulation OT Causality in Physical Mechanisms', Proc. IJCAI, 1977.
17. Stephanopoulous, G., Chemical Process Control, Prentice Hall, 1984. )
18. Underwood, W., 'A CSA Model-Based Nuclear Power Plant Consultant', Proc. AAAI, 1982.