PANEL ON DEALING WITH UNCERTAINTY

Participant ,;

Shigetoko l<aihara Universtty of Tokyo Hospital Tokyo, Japan

Edward Shortlifle Sl anford University Medical Center Stanford, CA, USA

Lolli Zadeh University of California Ber keley, CA, USA

lntrodudion

The problem of dealing with uncertainty arises in many areas of AI r esearch and has been attacked with a number of mechanisms. This panel is intended to bring together several of the researchers who have had direct experience with the Issue, to report br iefly on their efforts and discuss the relat ive meri ts of the various approaches.

The panel, originally entitled "Reasoning with Uncertainly", was r en amed to encour11ge a broader view of the topic . "Reasoning", i.e., infert:·nce, in the lace ol uncertainly is only one of the manifestations of the problem and may as a result be too narrow a perspecli·Je. Part of the emphasis in the discussion will be to examine the utility of a broader view.

The problem

In the few domains where concept s and inferences can be totally formalized (e .g., pure mathematics), there is no uncert;~inty of the sort we are discussing here. Unfortunately, In all real problems attacked by AI, we eventually run afoul of the issue. The problem commonly has two sources. First, the basic datil we have to deal with may be uncertain. This may occur due to noise (as in speech or vision), due to the fall ibility of our me;~surement procedures (as in medical laboratory

PANEL ON DEALING WITH UNCERTAINTY chairman Randall Davis MIT

Cambridge, MA, USA

tests), or due to the inherently ill -defined nature ol the concept involved (as in "block A is 'near' block 8"). Second, the processing ilpplied to that dal a m11y introduce its own uncertainly, typically because we lack a comprehensive understanding of the domain. In speech understanding, for instance, we have as yet no theory which allows us to proceed unerringly from waveforms to phonemes to words. AI best we can offer plausible guesses at each stage, and must hedge our bets by making multiple such guesses.

The Questions

To provide 11 focus for the discussion, the participants were asked to consider each of the following questions.

Constwing the issue of tmcerfainty in the broadest sense, what l<.inds of problems aftac~ed in AI appear to have dealing with uncertainty as a truly central issue? How is uncertainty manifested in ll1ose problem areas? For eltample, worll on deductive inference {6], modelling some aspects of common sense reasoning {9), hypothesis formation {5]. and signal understanding {1] all had to deal with uncertainly of sliglllly different form, and all developed different mechanisms.

The psychCl!ogical connection: What ~ind of behavior do people display when dealing with uncertainty? What can we learn from this and ought we to try to mimic if in our programs?

What are the current interesting and unsolved Issues In representing and dealing will! uncertainly?

What would constitute a good solution to the problem? That is, what performance characteristics would if display?

Has there been too much attention paid to the numbers? All of the schemes developed so far eventually use some numeric representation of certainly, but in some the computation and propagation of those numbers is the central concern of the scheme. As we note below, it might be argued that this is an undesirable situation.

Of these queslions,lhe last appears to be potentially the most controversi;~l; it might prove inleresl;ng to focus on it In more detail.

As lnspir a lion, consider the following argument that strong emphasis on the numeric a5peds of the problem is misguided. First (perhaps not surprisingly) none of the s.chemes developed to date has a precise notion of uncertainly, a situation which is manifested in the ill-defined nature of the numbers involved and the processing applied to them. AI best, we are given rough guidelines for what the numbers are to mean and some vague as-:;ur ;~nces that they can be as~. igned consistently. The arithmetic then done on them is typically ad hoc and its formal properties (e.g., its relation to other models of inference) often not well understood. The overall effect -- ad hoc processing of ill-defined numbers -- docs not inspire confidence in the result .

One common response is that not much attention is paid to the precise result, only the range it fall s in or the overall ordering it produces. This brihgs up yet another problem, a by now traditional dilemma: if the systems buill around these mech;~nisms are too sensitive to the numbers produced, that's bad becau<.E' no one is ready to defend the precise numbers. Yet if the !'y~.tems are relatively insensitive to the numbers, then all of that mechanism appears spurious. What is the "appropriate" level of sensitivity?

In more eener01l terms, I suggest as a point for discussion the following claim: The numbcn and numeric calculations involved in repres£'nting and dealing with uncertainly <~re at best artifacts. Any lime the main locus of attention is on the numbers, that's a symptom of something wrong, most likely that there is too little power in the underlying approach to the problem. As a contra~!, consider the HEARSAY-II (I] work on speP.Ch understanding, in which dealing with uncertainly was viewed as an issue of problem solving, and in which the system architecture and problem solving paradigm were designed to confront the issue. Various measures of uncertainly were introduced (e.g., the cr£'dibilily ol .hypolhescs, the focus of attention evaluations, el<.), but they played a far less central role bee au-:;e the basic approach -- which included the use of mulliplr cooperating knowledge sources, best-first search, etc., -- was itself cap<Jble of reason<Jble performance in the face of uncertainly.

As a spP.Cific eKample of the difference in methodology, consider HEARSAY's use of word hypothesis credibility ratings. One way to attempt to gel hip,h performance would have been to put effort into developing a finely tuned rating scheme which could be relied uron to give the best score to the corrcc I hypollu·~.is. But this wa-:; not done. Instead a version of besl-lirr,l !'E'arch w<ls developed which first sought out multi -word hypotheses and then used these as "islands of reliability" from which to prl.'ss forward the search lor a good global hypothesis [2]. The power of the system came from its basic probl£'m solving paradigm, not an elaborate arithmetic model of uncertainty.

Other mechanisms with similar abilities to deal with uncertainty have been explored in other problems:

partial mafcl1ing, in which the result of the match is not just success or failure, but a symbolic indication of how closely the match was :~chieved and what parts failed to match;

"constraint ~ll.lring", resolving uncerf;linfy by faking advantage of ways in which par fiat soluf1ons to 1/~e problem mutually consfr ain each other. That i~, rather than attempting to resolve every ambiguity as if arisE's by carefully ranking every slfcr native, ll!e prog1 am confmucs processing and uses other results to rule out ~ome po~:sibilifies. The feclmique has its roofs in relaxation techniques developed for solving differential equations, but has since been applied in symbolic form to a range of Ill problems (e.g., (8,3,7,4}).

The success of such mechanisms on interesting problems and their deempha-:;is of numeric propagation schemes is, or should be, food lor thought.