Expert Systems: How Far Can They Go?

Panel: Expert systems: How far can they go?

Terry Winograd, Randall Davis, Stuart Dreyfus, Brian Smith

We arc in the midst of a great wave of enthusiasm about the potential for expert systems in every area of human life and work. There is no agreement, however, as to just how much they can do, and where they will run into fundamental Iimits. The intent of this panel is to present and discuss some basic questions as to what expe1t systems can really be expected to do:

What is the nature of the problern domains in which expert systems are likely to succeed and those in which they will not? Are there domains in which their use might be dangerous?

How will their performance compare with that of human experts in the domain? Are there different facets of expertise that are not an1enable to pro· gramming? How can human and machine expertise best be combined?

To what extent can we count on rule·based systems for 'flexibility' in dealing with unexpected situations? How reliable will such systems be in cases where the programmers (or knowledge engineers) did not anticipate significant possibilities?

How can a 'knowledge base' be subjected to standards of accountability? Who is responsible for what an expert system contains and what it does?

Expert systems: What to do until the theory arrives

Randall Davis MIT

Jn reading a newspaper recently J was struck by the profusion of expert advice available. There werc three different expe1t opinions on the future course of the cconomy, scvcral compclling (and contradictory) opinions about the likelv course of events in the midcast, and a number of suggcstions about avoiding heart disease. as weil as claims about long term weather patterns, advice from Ms. Manncrs on behavior and guidelines from a therapist on drinking and sex.

All ofwhich made me begin to wonder:

Experts: How far can they go?

We are in the midst of a great wave of enthusiasm about the potential for experts in every area of human life and work. There is no agreement, however, as to just how much they can do, an.d where they will run into fundamental Iimits:

What is the nalure of the problern domains in which experts are like/y to succeed and Lhose in which they will not? Are there domains in which their use might be dangerous?

To what extent can we count on carbon·based systems for 'jlexibility' in dealing wilh unexpected situations? How re/iable will such systems be in cases where their teachers did not anticipate significant possibilit ies?

How can a person's know/edge be subjected to slandards of accounlability? Who is responsib/efor what an expert contains and what I hat person does?

Hardly an original satire, but it does serve several purposes. First. it demonstratcs that the questions are ncithcr unfamiliar nor inherently mysterious. The answers for people may not bc weil established, but we do know something ofhow to proceed and we do believe there is no magic here: some form of knowledge accounts for an expert's competence. The nature and source of it may be far from understood, butthat doesn't make the question unanswerable.

Next, it sets thc argument out on what 1 believe tobe an important dircction: the qucstions wc are asking are first about knowledge and only then about tcchnology. TI1at is, the first question should not be What can expert systems do? but rather What do we know? Only then we can address the technology issuc and ask And how easily can we encode thal know/edge? Both matter but the order is impmtant

Finally, the comparison is valid and provides an interesting way to proceed. Asking the sarne questions about people provides a useful, non·threatening way of examining the topics. Our answers may differ for peoplc and programs, but even those disparities will prove interesting. We explore people's understanding of a subject with a test that examines only a limited sample of their knowledge, and then extrapolate, saying that people who pass 'understand' the material, meaning by that something more than that they can do exactly the problems chosen for the exam. If a program passed the same exan1, would we be willing to do the same extrapolation? If not, why not? lf we can dctermine what it is that makes us hesitate in the case of the program, we have the beginnings of an intriguing research agenda.

In developing these themes I will argue that there are two attributes of expert systems (and much of Al} that are ccntral to this discussion:

It is a weak technology

lt is a technology for dealing with incompletely un­ derstood ideas.

I will suggest that the first of these is a temporary vice that will be remedied in time (though not soon) and that thc second is a permanent virtue.

Both of these have interesting implications for the use of expett systems (and indecd much of Al). Perhaps the most imporlant implication is thal most traditional rule-bascd expett systems will nevcr have all the knowledge they need, and as a consequence they are guaranteed to fail occasionally (though perhaps in­ frequently) du ring all oftheir operational.Iives.

I will suggest ways of proceeding that take th·cse issues into account, allowing us to cmploy thc technology while reducing the potential di rlicultics that can arise.

The nature of expertise

Stuart E. Dreyfus

Univcrsity ofCalifornia, Berkeley

All AI work with the exception.of a few 'connectionist' theories assumes that knowledge must be represented in the mind as symbolic descriptions. Expett-system builders fUtther assume that the expert possesses a particular kind of symbolic description: a knowledge base oF Facts, beliefs and 'if-then · ru I es that allow thc drawing oF inferences.

I will argue that expert-system builders Fail to recog­ nizc the real character oF expert human understanding. Expettise is acquired in a five-stage process. The beginner applies rules to contex.t-Free Features as would an expert system deprived oF situational knowledge appropriate to the particular case. The advanced beginner learns from experience to recognize aspects oF a situation without requiring a definition oF them in terms of context-free Features.

Aspects are rccognized after seeing several examples, apparently because oF their similarity to already experienced prototypical cases. An interactive expert system could use aspccts if they were identitied for it by a human user. At the next stage, the competent perFormer organizes behavior by selecting plans, goals, or per­ spectives which determine hierarchically what Facts to consider and what rules to apply. Expert systems can do likewise and, ifthcy acccpt human situational assessments, could appropriatcly be callcd 'competent systems.' This, however, is thc best they can do. The fourth stagc, proflciency, is achicved when the pcrformer no Ionger uses his knowledgc to sclect a pcrspectivc or goal, but simply recognizes the appropriate one based on prior experience in similar situations in which goals were chosen and evcnts either confirmed the wisdom of the choice or showed it to bc mistaken. As with the recognition of situational aspects by the advanced beginncr, thc involvcd, intuitive recog­ nition of similarity of wholc situations is apparently not produccd by rules operating on featllt·cs but sccms tobe effortlcss and holistic. While thc proficient performcr still analytically figures out what to do oncc thc situation is intuitivcly undcrstood, at the highcst Ievel of skill the expert has experienced so many situations that he associates with each prototypical situation in his mcmory the decision, action or stratcgy that he has found to work. He reasons out ncithcr stratcgy nor action. lntuitivcly responding to situational patterns as experience has shown appropriate, his skill depends neither on problern solving nor planning. Expert systems can neither recognize situations holistically without analysis into components nor know what to do without applying rules to decomposed knowledge, so they can be neither proficient nor expert

If time pe1mits, an expert will deliberate about his intuitive understanding. To fine-tune his responses, he will attempt to take account of subtle differences between his current Situation and similar prior ones, he will ask hirnself whether there might be another quite different way of intuitively viewing his circumstance, and he will consider whether he has had enough expcrience in the particular kind of situation to tntst his intuition. But he will rarcly regress to competent detached problern solving.

While 'competent systems' have their useful place, there is no reason to expect them to perform as weil as experts who have passed beyond the use of facts, belieFs, and rules of inference and who rely on memories of thousands of concrete experiences and what has worked in each. An examination of the performance of various expert systems supports the above analysis. In domains where human beings pass From reasoning to recognition as they become experts, expe1t systems, even when expetts participate in their development, never perform as weil as experts.