Explicit vs Implicit Profiling - A Case-Study in Electronic Programme Guides

1 Introduction

Recommender systems combine user profiling, machine learning and information retrieval methods to help users lo­ cate information by learning their preferences and making personalized recommendations to individuals ([Hammond et a/., 1996; Balabanovic and Shoham, 1997; Smyth and Cotter, 2000; Good et a/., 1999; Goldberg et a/., 1992]). There are two basic profiling strategies. Explicit profiling means asking the user direct questions about their prefer­ ences, usually in the form of an item rating; e.g. in PTV­ Plus (www.ptvplus.com), a TV show recommender [Smyth and Cotter, 2001; 2000], users rate individual shows on a five point scale from 'strong dislike' to 'strong like'. Ex­ plicit profiling has been used in a range of scenarios, from movies [Resnick eta/., 1994] to jobs [Rafter eta/., 2000], and the resulting profiles are assumed to be reliable indica­ tors of user preferences. However, users are lazy and do not like responding to explicit requests for feedback. As a result explicit profiles often contain few ratings, which can limit the degree to which they will accurately reflect a user's changing preferences over time.

Implicit profiling methods construct user profiles by infer­ ring user ratings from so-called interest indicators based on user's interactions with the system [Claypool et a/., 2001; Goecks and Shavlik, 1999]. For example, [Morita and Shin­ oda, 1994; Konstan eta/., 1997; Claypool eta/., 2001] re­ port a positive correlation between reading times (for Usenet News and Web pages) and content ratings. [Claypool eta/., 2001] shows a correlation between content ratings and both mouse movements and scrolling although mouse movement was only found to be a useful rating predictor at lower rating levels. Interestingly, [Claypool eta/., 2001] finds no correla­ tion between user ratings and the number of clicks to links on a Web page.

In this paper we seek to evaluate the use of implicit profil­ ing in a very different application domain, Digital TV (DTV). DTV users are finding it increasingly difficult to identify rele­ vant TV shows among the hundreds of available channels and recommender systems such as PTVPlus have been forwarded as a potential solution [Smyth and Cotter, 2000]. PTVPlus uses its recommendation engine to generate a set of TV pro­ gramme recommendations for a target user based on their profiled interests using a combination of collaborative and case-based strategies, and presents these recommendations in the form of a personalized programme guide. So far research in the area of personalisation and recommendation within the DTV domain has focused almost exclusively on the use of explicit profiling. Consequently, the key question we wish to answer relates to the type of implicit interest indicators that can be identified within the DTV domain and the extent to which they can accurately reflect a user's true preferences.

2 Implicit Ratings in the DTV Domain

In this research we use the Fischlar video library system [Lee eta/., 2000] developed by the Centre for Digital Video Processing at Dublin City University (www.cdvp.dcu.ie). Fischlar has been fully operational for a number of years and attracts hundreds of users on a regular basis. It is a video li­ brary system that allows users to record, playback and browse TV shows through a normal Web browser. Users browse personalized television schedules, provided by the PTVPlus personalization engine [Smyth and Cotter, 2000], to select programmes to be recorded by the system. TV shows are captured in MPEG-1 format and processed in order to index their content for storage and to extract key-frames for content­ based browsing. Fischlar users can select programmes to record up to 2 days in advance of broadcast and can access previously recorded programmes for playback or browsing.

As Fischlar takes advantage of the PTVPlus personalized TV listings service, there exists the ability to capture explicit ratings feedback for its user profiles (Figure 1) using the rat­ ing icons beside each programme's description. However, unlike PTVPlus, Fischlar users can interact directly with programming content, and this provides new profiling opportuni­ ties. In particular, the record, playback and browsing actions of users are suitable as implicit interest indicators; e.g., select­ ing a programme to be recorded is surely a strong potential indicator of interest. Thus, Fischlar's profiling capabilities have been enhanced to capture implicit as well as the explicit profiles, by logging all record, playback and browsing actions (Figure 2).

3 Experimental Evaluation

Our aim is to evaluate the usefulness of Fischlar's implicit user profiles relative to its explicit profiles. In particular we wish to determine whether or not these implicit profiles are any more or less accurate at predicting user preferences than the explicit profiles gathered directly from user ratings.

3.1 Test Data

Our experiments draw on 650 live-user profiles from the Fischlar system. In particular we make use of 5 different types of profile:

1. Explicit profiles contain explicit ratings that Fischlar users have provided;
2. Play profiles contain only those programmes that a user has played back;
3. Record profiles contain only those programmes that a user has recorded;
4. Browse profiles contains only those profiles that a user has browsed using Fischlar's key-frame browser;
5. Combined profiles are a combination of play, record and browse profiles.

3.2 Algorithms

In addition we make use of two different programme recom­ menders:

1. CF is a traditional collaborative filtering recommender as described in [Konstan et al., 1997; O'Sullivan et al., 2002b]; it finds similar users by calculating program overlap between the target user and all others, and then recommends to the target user programs from the most similar users that they have yet not rated.
2. Sim is a similarity-based recommender that draws on similarity information mined from user profiles as de­ scribed in [O'Sullivan et al., 2002b]. It uses data mining techniques to find relationship between items and then harnesses this information both in profile similarity and recommendation ranking (ranking all possible recom­ mendable programmes) to recommend programmes to the user.

Due to space constraints, it is not possible to discuss these recommenders in detail; the interested reader is referred to [O'Sullivan et al. , 2002b; 2002a]. Suffice it to say that they each draw on a fundamentally different recommendation strategy, be it collaborative as in the case of CF, or content­ based, as in Sim.

3.3 Method

First each profile is randomly split into a training profile and a test profile; this is repeated for a range of different split-ratios for each profile. The training profiles are used as the basis for recommendation; in each case we make 50 programme recommendations for each user. The quality of the result­ ing recommendations is evaluated with reference to the cor­ responding test profiles by computing the percentage of rec­ ommended programmes that appear in the target test profile. So, for example, we use the training play profiles to gener­ ate recommendations that are evaluated with respect to the test play profiles; similarly for record and browse profiles. In this way we can compare the quality of the recommendations generated using implicit play, record, browse and combined indicators to the quality of recommendations generated using explicit ratings profiles.

3.4 Results

The results are presented in Figure 3 as an accuracy bar­ graph for the various different groups of explicit and im­ plicit profiles (play, record, browse and combined). In each grouping we present the accuracy results obtained by the CF and Sim recommenders. For example, we find that the ex­ plicit profiles deliver recommendation accuracies of2.5% and

15%, for CF and Sim recommenders, respectively; clearly the enhanced Sim recommender offers significant benefits over the standard CF recommender as previously discussed in [O'Sullivan eta!., 2002b]. Each of the implicit profiles gen­ erate consistently higher recommendation accuracies for both CF and Sim recommenders. For instance, in the case of the play profiles, the recommendation accuracy is 9% and 30% for the CF and Sim recommenders, which is a significant in­ crease on the explicit profile accuracy, especially for the Sim recommender. In fact, the implicit profiles yield recommen­ dation accuracies in excess of 20% for the Sim recommender compared to the 15% obtained for the explicit profiles.

These results indicate that playback, recording, browsing and combined behaviours in Fischlar serve as competent in­ terest indicators when it comes to profiling user preferences. In each case we find an increase in the quality of recommen­ dations made from profiles containing implicit information. Moreover, the scale of this increase is largest in the case of the Sim recommender, due to the fact that this recommender directly mines the profiles in order to generate programme similarity knowledge as the basis for recommendations. The availability of the higher quality implicit profiles helps to im­ prove the similarity rules used by Sim when compared to the similarity rules mined from explicit profiles.

4 Conclusions

We have evaluated the use of implicit interest indicators as the basis for user profiling in the DTV domain. Research in more traditional domains, such as Web browsing or Usenet News, indicates that some implicit interest indicators (e.g., read-time and mouse movements) are capable of serving as reliable alternatives to explicit profile information such as user ratings. Our research indicates that the same is also true in the DTV domain. We have compared the quality of recom­ mendations generated from explicit and implicit profiles and found, through the use of a deployed video library system, that in each case the implicit profiles are capable of generat­ ing recommendations that are as good as those generated by the explicit profiles.

Acknowledgements

The support of the Informatics Research Initiative of Enter­ prise Ireland is gratefully acknowledged. The support and advice of the Centre for Digital Video Processing at Dublin City University is also gratefully acknowledged.