An Analysis of Psychological Experiments on Non-Monotonic Reasoning

1. A CASE FOR NON-MONOTONIC REASONING IN HUMANS: 'DEBRIEFING' AFTER DECEPTION EXPERIMENTS.

A speaker at a conference may be heartened afterwards by someone from the audience who congratulates him for his interesting and clear exposition. If later he fmds that this person mistook him for a potential referee, his self-esteem may decline again somewhat but it will probably not sink all the way back to its original level. In contrast to the AI examples of belief revision, people are often reluctant to adjust their opinion after the original evidence is descredited. This phenomenon has received special attention in connnection with psychological experiments in which subjects are deceived about the true nature of the setting, and are later 'debriefed' about the manipulation [Baumrind, 1964; Holmes, 1976; Ullman & Jackson, 1982]. The subject must be convinced 'that the information was fraudulent, i.e. he must be dehoaxed. Sometimes, the subject's feelings about himself (e.g. due to having behaved unethically [Milgram, 1963]) must be altered This aspect concerns the subject's behavior, which cannot be refuted. Dehoaxing on the other hand, concerns the experimenter's deception. For this aspect, conclusions accepted earlier during the experiment can be disproved. Therefore, in the context of non­ monotonic reasoning the dehoaxing aspect of debriefing is the more appropriate one to study.

1.1 EXPERIMENTS ON DEHOAXING PER SE

The aspect of dehoaxing would be difficult to isolate from the experiments, since these differ greatly in the nature and degree of deception. Fortunately enough, many experiments have been conducted on dehoaxing per se, employing different designs, different domains, and varying degrees of external validity. A

sample of representative studies is summarized in Table 1. As the paradigmatic example I will use an experiment by Ross, Lepper, and Hubbard [1975], which was very carefully designed and has been replicated many times. In it, the subject was presented with cards containing pairs of suicide notes. She was told that one note in each pair was genuine, the other bogus, and she was asked to indicate the genuine one. In addition she was informed about the average .score in a pretest. The subject received false feedback indicating success or failure after each card. After completion of the task she was informed that the feedback had been determined prior to the experiment, and that it was not related to her actual performance (this was called outcome debriefing). Nevertheless, the greater the apparent initial success, the higher she estimated her scores for past and future performances. In snort, subjects showed a substantial perseverence of the initial, erroneous impressions. Only after the process underlying the perseverence was explicitly discussed was the initial perception abandoned (the process debriefing).

Ross et al.'s explanation for the phenomenon has essentially two parts. The fli'St part stems from the literature on attribution theory: An individual who witnesses a surprising (or extreme) outcome generates (searches for) confirmatory evidence capable of explaining the observed outcome. Second, if the original evidence ·for the outcome is removed, these antecedents may survive to give independent evidence for the outcome. For example, a subject may attribute her success on the discrimintion task to the fact that she was once personally acquainted with a suicide victim.

It may be argued that Ross et al.'s experiment does not in itself prove the presence of self-generated confirmation-biased evidence. Independent support for its presence has been found in various ways, however. For example, enhancing the possibility of producing such evidence increases perseverence [Anderson, Lepper, Ross, 1980]. On the other hand, when an interference task (e.g. counting backwards from 200 by 3) prevented the subject from engaging in explanations, no perseverence effect could be · established [Fleming & Arrowood, 1979; Barefoot & Straub, 1971].

2. A MODEL FOR THE PROCESS OF DEBRIEFING

2.1 DEBRIEFING MODELED USING TMS

To introduce the model for Ross et.al.'s experiment I will use Doyle's [1979] TMS, a technique that allows for non-monotonic reasoning···. In this technique every assertion entering the data base is represented by a node. A record is kept of the dependency of nodes on inferential steps, i.e. the justifications of a node. This

· ·

Table 2. The debriefing experiment by Ross et al. The colums labeled 'dependencies' show how beliefs depend on other beliefs. A black dot indicates the belief is IN for the situation before the experiment (I), after feedback (II), after outcome debriefmg (III), and after process debriefmg (IV).

... The example is too simple to justify a comparison with other techniques. TMS was chosen because it is the most general technique proposed to date.

way the inference steps can be retraced to maintain consistency in a system. An assertion that is believed is called IN. An assertion that depends on the fact that another assertion is not believed (i.e. is OUT), is called an assumption. In the debriefing example someone else's assertions are believed as long as it is not believed that the other person is lying. If the latter belief comes IN, the assumption will go OUT (is not believed anymore). The debriefing experiment is depicted in Table 2. with the different stadia in terms of TMS.

The subject starts out with no particular beliefs about the task. Wh~ the experi_m~ter says the subject has performed well (c), she mfers that this lS the case (b). From this she generalizes to the belief that she is generally good at recognizing real suicide notes (a). ~is can be probed, e.g., by asking a subject how she would sco~e m the future, or. how she thinks she compares to other subjects. At the same tune she generates confirmatory evidence (e), which comes IN. This evidence itself is an additional justiflc~tion for belief a.The.debriefin~ takes place by informing the subject about the decept10n (d). Smce b depends on d being OUT, b goes OUT when d comes IN. But when asked, the subject will still believe a, on the basis of the independent support e. The process debriefing consists of an elaborate discussion of the perseverence phenomenon itself. The subject becomes aware of the self-generated confirmatory evidence she used, and leaves this out of the argument, i.e. e goes OUT, and as a consequence (a) goes out as well.

There is more to say about factors that are conducive to belief perseverance (see e.g. [Schul & Burnstein, 1985]). Keeping thin.gs simple however, consider a variation of the experiment by Ross et.al. One could start the system with d IN. In other words, the subject is told in advance that feedback will not be genuine. What will happen? We will come back to this after I have taken a closer look at the states of belief involved in the experiment.

2.2 STATES OF BELIEF AS ADMISSIBLE EXTENSIONS

The model developed so far describes the intended behavior (i.e. in the Ross et al.'s experiment) by showing how the subject gets from one state of belief to another. To ensure that the system represents the intended model, however, it must also rule out behavior not found in (or falsified by) the experiments. A way to find this out is by examining what belief states the system is capable of generating. To this end I will cast Ross et al. 's experiment in the form of a non-monotonic theory. Remember that at the heart of the model lies the notion of 'assumption.' Specifically, the subject believes the experimenter in the absence of reasons not to. The reason for believing C in the presence of A and in the absence of B will be denoted as:

A II B II- C. If B is empty (0), the inference from A to C is like an ordin~ implication····. If B has the form 'Defeated(R)', the reason R 1s called defeasable. Let us describe the dependencies from Table 2 as a set of reasons R (indexed by consequent):

From a set of reasons a set of extensions can be derived. They are the analogue of the 'deductive closure' in ordinary logic, and represent the internally consistent beliefs. Computing the closure (e.g. [Etherington, 1987]) R* ofR gives two extensions:

which are precisely the statements believed before and after debriefing. Now, where does the process debriefmg come in? Statement a perseveres via rc, re and ra2. At least one of these reasons is apparently attacked by E (the experimenter). Reason rc cannot be refuted since c: is a fact. So, by discussing the perseverence process itself, the experimenter either defeats r~, or ra2· Let us ftrSt assume the fonner. This can be fonnallzed by rewriting re (and R changing accordingly):

·n·But note: a reason is used to justify a belief. It is not an inference rule. So while e in Table 2 is justified by c:, this does not mean it is logically implied by it; in general it isn't.

re1· c II Defeated(rel) 11-e

re2 · 011011- Defeated(rel)

Now, in addition to R1· and R1·, two new extensions result:

R3 · · R u {a, b, c, Defeated(relH

R.a ·- R u {d, c, Defeated(relH

where R/ gives the belief state after process debriefing. R 3 · shows the efficacy of the process debriefmg, i.e. as measured by the subject's prediction of her future perfonnance on a similar experiment. Another interpretation is that a subject may be forwamed not to generate confmnatory evidence, i.e. to have re2 ready in advance. A natural settin$ where this could occur is the courtroom. Indeed, in such a situat1on subjects are much easier to debrief [Hatvany & Strack, 1980]. Independent support to propose rel and re2 stem from the experiments with an interference task. The interference effectively blocks the generation of confinnatory evidence, or fonnally, defeats re 1· In this case R 3 · and R4 · represent the states of belief before and after outcome debriefmg in the interference task. Now re1 and re2 have been sufficiently justified, it remains to discuss the role of r al· It could be that this reason is defeated during process debriefing, although this cannot be ascertained on the basis of the experimental evidence currently available. In any case, it can be fromalized in a manner analoguous to our treatment of re above.

2.3 THE MODEL fi NEmiER TOO WEAK NOR TOO S111.0NG

Since the model proposed above is based on an existing fonnalism for non-monotonic reasoning, I want to relate it to a criticism that has been advanced concerning such fonnalisms. Recently, Hanks and McDennott [1986] questioned whether these fonnalisms produce the expected results. They provide axioms for a simple problem (the 'Yale shooting scenario') and show that a well-established technique (c.q. predicate-circumscription) produces not only the intended extension, but in addition one that 1S counter-intuitive. Now, whereas Hanks and McDennott could have chosen between attacking either the axioms or the inference technique, they chose the latter. For this reason, in the section above I generated all the extensions of the proposed axioms for the Ross et al. experiment, and checked if they indeed belonged to the states of belief I wanted to model. They did. So the model is guaranteed neither too weak, nor too strong in generating states of belief. Yet, this is not enough to ensure the same holds for the intended behavior, i.e. for the sequence of states. To see this, suppose in the Ross et al. experiment the subject is briefed in advance that the feedback will n~t be genuine. That is, we start in Table 2 with dIN. Following through the experiment we will see ·that the same behavior ensues as before. In other words, the subject believes she perfonns well on the experiment even knowing beforehand that the feedback is bogus. This surely runs counter to our intuition. A similar reasoning as in the 'shooting scenario' therefore leads us to believe that our model, as defined by R, is too weak (it predicts unintended behavior). Yet, let us stay in the vein of this paper, and see if the prediction can be tested. In fact this has been done already by We~er, Coulton apd Wenzlaff [1985] who briefed the subjects in advance with the same words that were used by Ross et al. during debriefmg. They found the same perseverence phenomenon, on the basis of which they rejected the theory of Ross et al., and fonnulated a principle of transparency of denial. This principle basically says that when people encounter denied infonnation, that infonnation is available despite the denial. However, in their experiment Wegner et al. tell the subject in advance that the infonnation she will obtain is false, i.e. the infonnation is not available at that time. In my opinion it is not necessary to introduce a new principle. Using our tenninology, Wegner et al. seem to think they defeat reason r el· whereas in fact they produce d, so that b cannot be derived but c can. Whichever may be the case, the experiment confmns the counter-intuitive behavior predicted by our model.

3. CONCLUSIONS

People often cling to their initial beliefs more strongly than appears warranted. Based on a wide variety of experimental settings, this paper undertook a fonnalization of this phenomenon. It may contribute to psychology and to AI in the following way: For psychology:

1. It fonnalizes and specifies experiments in the debriefing paradigm, and thus
2. It offers a better framework for analyzing these experiments and comparing explanations

ForA/:

1. It demonstrates how truth maintenance techniques and non­ monotonic theories can be used in modeling experiments on belief . revision
2. It calls attention to the role of self-generated confmnatory evidence as an important factor in human non-monotonic reasoning
3. It demonstrates that intuitive appeal of predictions is not a reliable criterion for evaluating a descriptive model.