Building and Exploiting User Models

1. Introduction

As comput~r·. rom~:! to bt> ll';cd by a larger number of people to help perform il p,real variety of tasks, it is becomming more .:md more important for them to be easy for people to u·.c . 1 here are many factors that can contr ibute to the ea~.e of u~.e of a computer system, ranging from the good dr·.ign of terminals to the speed of the sy stem's re·.pon·.e, the l'lppropriiltencss of its response, and the nilluralnesr, of its input and output languages. Yet .:molh£>r filclor that can significantly affect the ea·,e with which people can u5e a system is the ability of the sy51em to tailor its behavior to the specific neecl ~; of an individlial u~.er . II is this aspect of the habitability problem th<Jn will be discussed in this paper.

In order for a <:.yslcm to be> able to respond to the individual ncecb of it ~. w.cr r. , it mur.t have access to some kind of model of e<1ch u·.er. BecillJ!;C the term "user model" can bf.' u~.ed to mean several different things, it is imporli1nt to specify what we rncan by it. The three major dimensions along which user models

can be classified arc :

• Are they models of a c;H10;1ical user or are they models of in(lividual users?
• Arc they constructed explicitly by the user himr.elf or are they ab!.tracted by the system on the bar.is of the user's behavior?
• Do they contain ~.hort -lerm, highly specific information or longer - term, more general information?

'Thio rofto~rch wan nponn'""" on pnrt by the Oof8nGft Advanced Reeurch Projoch 1\enncy CD00), 1\RI'/\ Ordor No. 3597, monitored by the Air Force 1\vionocs I ~hnr~tory Unrl~r Contract F3:lGIS-7R-C-ISSI. The author wu avppart~d by ~ '-'""t from the Xerox Corporation

The views and concfuoio>no cnnt ainrd in thos documont art those of the autt.or and should not hn intNprrted ·· reproaenlinc the off icial policiee, either exproooed or impHrd, of the Oefenoe Advanced Rtaoorch Projecla Acency or tt.e LIS Government

The work described here has concentrated on user models in one corner of this space. Models of individual users were buill becau<:.c of the facility for personaliz<~tion they provide that i~ lacking in a canonical llser model. Implicitly con~. trucled models were used bcrau~.c of the inherent inaccuracy and the annoyance of requir ing u!.crr. to con~. truct their own models of themr.~lvcs . The choice of posi tion along the third dimcm.ion wa·. much l e~.s cleil r cut. EJot h short-term knowlcdgr> (r.11r.h a·. the! topic currently being discussed or the goal now being punued) and long-term knowledge ( r. uch as the level of the u~er's knowledoe about the probiPm domain or (\O<~I s that arc likely to be pursued) can be imporlilnl to the perform11nce of an interactive system. Out it .:~pp,..arcd likely !hill different techniques would be appropri ;ll~ for dealing with the two types of 1-.nowledp,e. Models of long- term user characteristics were chosen bcc .:~use very l ittle work has been done with them, while short - term u ~. er models were already being explored, part icularly in the context of natural langua!',e undar~. tanding . So the rest of this work will deal E>xplililly with the i·;suer, involved in individual user, implicit, lonr, - term u5er model ing.

2. Some Requirements for an Individual-User, Implicit, Long-Term User Modeling System

The ffiQd.e[ mur.t be Q!!ill. cl_yn<~.rn ic aiJ:i, as the ~stem is intera~!J.~ ~..Uh !IJ.c ~ ~ r~eL ~nc! ~~rJ_qn.r·JDr: Ll§. !2s'k: Because the rnod~ l i-; to be l)llill impl icitl y ilnd bec ause users do not want to be forc ed to an~.we r a long list of questions before they can brr, in to u~e the system, it is necessary that th.- ~.y ·, lf>rn be <Jblc to exploi t a partially constructed user modr~ l i1nd lh<'ll it be able to tell, as it is performing its · m.:~jor I ,, ~. k. when i t is appropr iate to update the ll', cr model ;,nd how to do so.

The ~:.!.~!:'1 ~· 1~9_111<1 ll.'?. ~f)IC' ! ·~ ~nrr~ !!.~ ~J~.h h,r_o~~2r.~ about tht;!. ~·.~· C':!_ !rom !' ·· ~ little. i.n!O!!:f:l <~tl<m !"i !1._0Ssiblc_, This is import i'lnt if I he u~. cr i ~ not to be required to answer a large number of ques tions about himself. The major technique that can be exploited to do thi5 is the use of stereotypP.s. A ~.tcrr.olype can be activated on the basis of a single action and can suugest a large array of likely th;~r;~ctr.ri<:.litr. of the ur.er. Stereotypes serve a function very simil;~r to !hill of scripts [2); they group together frequently co-occurring events to provide usE!ful prerliclion!. lhiln c;~n guide the system.

The US~!:. mo.d~l.l!}P. ~;y!;f .c!l'_ r<ll1~:1 !?.~:!. !!_ble tQ fs>.P_~ will"!_ the uncertainlY. lhill will Mise from the fact that most of wllill i! geii~:YQ.i". ~hQ"!{ !!l!i ir.::~!. li·-;.-ciillv U:,st ! @Qrt guess. Bccau!>e most of the knowledge ;~bout the user will be based only on likr.ly inferences drawn from his behavior, the 5yslrm mw.t ;~ssocit~le with those inferences some me;1~' ure of how confident it is of them. It must also remember the bil~.is for each inference so that when conflicts arise (ils they are sure to do), it will be able to resolve them in !;Ome reasonable way. Of course, to do this, it must also possess some conflict resolution rules. In this respect, user modeling systems are similar to ol her prob;~bilislic reasoning systems, such as MYCIN (3).

!! must be l?.9X·ihlc_ !.<:?. ~oclify !!!g. dilla base Qf. stereotype_s.. ~Q. !.11!!.! !hey. y.rill ~-'L~IJ!~!~I!Y. £harJls.!i!.rl{~ the sysl_g_f!!~ !J~.~r~~ €'~· ?'CClJr..!1)c~y !J.!~ p_Q.f:r:iblf!, Since it will almost never hP. po!.~.ible to con!.lruct stereotypes that are completely ilccur;~te and since the stereotypes are so imporl;ml to the function of the system, it is important that the sy~dem monitor their accuracy and change both them and their lr iggers as warrenled by the actions of the uscn.

3. A Brief Description of Grundy

In order to have a forum in which to explore thesP. issues and to be able to make a convincing case that some solutions hild been found, it was deemed necessary to tackle the is!.ues in the context of a specific task dom;~in and ;~duillly to build il ~.yslem into which the solutions to the issues could be incorporated. The task that wa!. cho·.en for this experiment is to suggest to users nov(>!<; !hill thf)y might like to read. This is an appropri;~te dom;~in since a wide variety of people read novels, hut they do not all like the same ones; and there i!. enour,h flexibility in the task that knowledge about the individu;~l user can be exploited.

The novel recommending program, called Grundy, first asks one or two q11r -..tions, on the ba!;i·. of whith it begins to build its model of the user, usu;~lly by evoking one or more stereotypes. Then it begins the proces~ of recommendine, nov~:+.. Eilr.h lime it selec.ls a book, it asks the u~.er whether or not he hils reild it. If he ht~s, he is then a~~.ed whether he li~.ed it. II he did, Grundy can update ih. model of him on the ba!.is of the known characteristic.!. of lh~ book. If he did not li~.e the book, Grundy atlemptr. to find out why not and then to update its model of the u~. er ;~ppropriillely. II, on the other hand, the user hils not reild the boor., then he is given a description of it and then ;~sr.rd whether he thinks he would like it. If he think!. he would, Grundy need do nothing -- it ilppcars to be on the right lrilck. But if he thinks he would not likr. it, Grundy must try to discover why not and then to updille it!. model of him. Thus, as Grundy interacts with the u!.er, its model of him will evolve and begin to coni a in ~pccific knowledge ;~bout him, infered on tile bi!·.i!. of his response~, in addition to its early beliefs, which were principally based on the prediction-;; of stcreot y pes.

4. The Performance of Grundy

In order to collect evidence lh;~l Grundy is succussfully exploiting user models, lwPnly-three people were observed using the sy~. tem. Several aspects of Grundy's performance were investigated.

4.1. Grundy's Succo~:s at Recommending Novels

Although buildinp, the ideal novel recommending program was not the primilry goal of this research, it is import ant to est ablio::h I hill Grundy exhibits some form of intelligent ;~clion in order to show that the u!.er modeling techniques arc effective. In order to measure the system's succe!;s, il small experiment w;~s conducted at the end of e;~ch convcrr.;~lion Grundy h;~d with a user. Some books were seiC'clcd r;~ndomly from the data base and then sueee·.;led to the u!.er. It was then possible to compare Grundy's rille of r.uccess at recommending books that looked good to the users both when it was exploiting the user model ;~nd when it was not. Figure 1 shows the results of the comparison. It shows the number of good suggr.~. tion!. (i.e. the ur.r.r silid he mieht like) and the number of bad ones (i.e. the user said he would not like) in both thC' controlled and random modes. The datil ciNrly demonstrate that the user models do contribute !.ignificantly lo Grundy's performance as a novel recommender.

4.2. Grundy's Succes!l at Buildinr. Models of Its Users

Several ;~!tempts wer·J m;~dc to lest more directly Grundy's succes·. at huildinr: ml)dels of its users, by comp;~ring the mon"h h11ill by Grundy lu self-descriptions provided by the uc;ers. This is not a perfect measure of Grundy's ~.uccess, however, because people's de~. c. riplions of thrm~.r!lvcs do not tend to be very accurate (or even con!;i!.tenl). Despite this, the data indicate that Grundy is doing quite a good job at building models of it s users. This conclusion is not, of course, surpri!.ing in lir,ht of the data on Grundy's success at recommcnrling books.

4.3. Learning in Grundy

As has already been disc u~.~. ed, it i~ import ilnl that the stereotypes and lrip;cers that form the bulk of the database of " U!·N modPiine ~.ydcm be ;~blf! to change to reflect the actual body of u!.ers of the overall system. On the other hilnd, it is important !hill no single user be allowed to have a momentous effect on a system's global view of the c la5s of e~ll users. So in running Grunrly with <1hout lwC'nly u~.cr!;, only the mo~l commonly invoked (;lercolyp0~. could . be expected to change significantly. The most frequently e~ctivaled stereotype we~s the MAN r.lcrl'olype, .1nd it did show interesting e~nd ~.ignific;~nl < hilnGe over the period Grundy was run. For exilmpiC', the stereotype initially predicted the~! men would lil'.r. books the~! are very fast-moving and full of ~. u·.pew.c. The values of both of those charac.lcri!;lic·. dccrca~.cd considerably on the basis of �rundy'~. experience with its users. Both of these changes can ea·.ily he exrl<~ined by the difference between the group the stereotype was originally intended to chMac.INi7e (all .1dult male Americans) and the group it wa-. adw1lly 11pplied to in practice (some adult male inlellec lu<tls). The people who used the system, although they we-re men ;~nd did r.ecm to like fast -moving, excilint: books, ak.o tended to like philosophicill novel~. ;m··l lilcr <~ry classic(;, which tend to be much calmer. lhu·. by tl1r end of the experiment, Grundy had a more accur;~le picture of the men it was encountering than it hild h<ld at the outset. This suggests that the learninr, mcch;misms are capable of causing at least some ~.oris of improvement in the stereotype dal a ba~.c of a user modeling system.

4.4. The Effectivene!is of Sloroolypes

In looking at the results of GHUNDY's attempts to usc stereotypes to build useful models of people, one is reminded of one imporLml chMarlcrk.tic of stereotypes and thus of sy!;lew; that use them. Although a stereotype may often provide highly appropriate and useful information about il person, it i!;, ill best, an expression of a tendency ;~nd not an absolute truth. One example will ~.erve to illu~. trale this point clearly.

Sever at stereotype~ ·. in �f~lJNDY predict that people who fit them will be interested in the conflict "art-vs-science". Tho~.c predictions caw;cd GRUNDY to believe, c.orreclly, that sPveral w.crs were interested in that conflict and lhu~. to recommend bool'.s involving it. But one per~.on, who fit three stereotypes !hill predict an interec;t m that conflict r.ilid, on the second questionnaire, that he wa~. definitely not interested in it. Figure 2 shows ;til the ~.iluillions in which GRUNDY believed the u·.er wa~. inlerc·,ted in art-vs-science.

5. Conclusion

If one were to dir.till lhi·. lhe-;i·. into its mor.t essential points, there would, I think, be three.

The first is the awarene·,s of the potential usefulness of user models of thir. ~orl. Thi~. awareness is just now becoming more important a:: the use of computers spreads further and further into the daily livt!S of more and more people. If lhi-:; lher,is does nothing except make people think about building these computer systems so thai they can better serve their users, lhi~ effort will not havr. been Wil~.ted.

But reali?.ine a neP.d ilnd ·.ali!ofying it are not the same thing. The second ~.ignifitanl point of this thesis is that, for at leilst one such ~.y~.tem, in ill least one domain, an individual u~·er model Wil~· made to work and to guide a performance system succcs!.full-,.

And yet the third conlrtbulion, after arguing for necessity, and dr>mOn!.lralme, fea·.tbtltty, i·, sur.ee~ling techntque. Not only arc u!.er rr.odels w.cful and doable specific techniques for .building them have bee~ developed. Although no claim i!. hcinr, made that these techniques arc the perfect one!., they hilve been shown to be workable in one domain and extensible to others.

These three poinlr,, the ur.r.fulncss, the feasibility, and the t~ch':'tque~ of u·.·~r modclinr,, constitute the major contrtbulton of lh1s thesi5.

A more del ailt"d di ~.cw. ~·ion of all of the iss lles that were mentioned in this Pilrer, M · well as several that were not, including a description of the actual mechanism!> employed, and a discu~. sion of how to extend the ideas to other domains, can be found in [.I).

Referem.es

1. Rich, E. A. Builclinf( attcl Exploiting User Models. PhD lhe~.is, CMU, 1979.
2. Schank, R. C. and R. P. flbleson. Coals, Plan.r, Scripts and Understanding; An Enquiry into /-Iaman K~towlcdgc Structures. Erlbaum Press, 1977.
3. Shorlliffe, E. H. MYC/N: ContfHttcr-bascd Consultations in Medical Consultations. . American Elsevier, 1976.