Information Filtering: Personal Profile Usable as Stereotype Profile?

Information Filtering: Personal Profile Usable asStereotype Profile?Corne VerslootFaculty of Electrical Engineering, Mathematics and Computer Sciencec.a.versloot@student.utwente.nlABSTRACTThis paper describes a research that gives an indication of theuse of personal profiling techniques for stereotype profiling. Astereotype might be seen as a ‘single person’ that represents agroup and so personal filtering might be used for stereotypefiltering. To test this, a group of 6 people used an adaptedinternet browser with a single personal profile and ranked a setof documents afterwards. The profile was also used by peoplewho did not fit the stereotype to test how vulnerable a personalprofile is for misuse. These tests show that the profile can filterdocuments for a stereotype group but also has difficulty rankingcertain documents.KeywordsStereotype profile, personal profile, information filtering,browsing behaviours1. INTRODUCTIONThe last few years the amount of data on the internet hasincreased exponentially. Searching all this information usingconventional searching algorithms takes a lot of time. A lot ofresearch is done on for example systems that can answerquestions using the internet and on systems that identify and useuser’s interests and preferences. In the latter ones, profiles areused to store the interests and preferences of it’s users. Ingeneral there are two different types of profiles which have a lotin common but also have some very important differences. Thefirst type of profile is often referred to as a ‘personal profile’and is used by one person. The second profile is used bymultiple users and is often called ‘stereotype profile’. Bothtypes of profiles filter documents using the interests of it’suser(s). These profiles try to learn these interests and try to keeptheir knowledge up to date.1.1 BackgroundThe main difference between those two information filteringtechniques is the way the profiles learn and the way the profilesare stored. One of the most important features of a personalprofile is to adapt to the user’s needs as soon as possible, likedescribed in [WIY99]. In this way the profile continues to be anaccurate representation of the user’s interests. Because there isonly one user involved the profile (in the most simple case) canconsist of a collection of important documents or words. Forstereotype profiles the aim is to represent the interests of anPermission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copiesare not made or distributed for profit or commercial advantage andthat copies bear this notice and the full citation on the first page. Tocopy otherwise, or republish, to post on servers or to redistribute tolists, requires prior specific permission.3rd Twente Student Conference on IT , Enschede June, 2005Copyright 2005, University of Twente, Faculty of ElectricalEngineering, Mathematics and Computer Scienceentire group of people. Logically it is not useful for such astereotype profile to adapt as fast as possible to changes of oneof its users because it might have unwanted effects for the otherusers. In time, a lot of different filtering techniques haveemerged. Research like in [KSS03] is done to comparestereotype filtering and personal filtering. In this particularresearch a rule based personal profile was compared with a rulebasedstereotype profile. The conclusion was that the stereotypefiltering outperformed the personal filtering.Despite these differences, both types of information filteringtechniques also have a lot in common. A stereotype can be seenas one person that represents a group of people. For this ‘single’person a personal profile should also be able to filter documentsaccording to the user’s interests. This paper describes a researchto find out if the learning techniques used for personal profilesare (in some way) useful for stereotype filtering.2. RESEARCH QUESTIONS ANDMOTIVATIONA good learning stereotype profile would be useful in a greatnumber of public places to search various kinds of informationsources. For example, people can use a stereotypic profile in thelibrary to aid them while searching for (what they believe are)good books. They would just choose an appropriate profile andstart searching, the profile would filter out the books which areprobably ‘not wanted’. The same can be done by internet searchengines. People can be assigned to a stereotype using theirprofession, hobby’s etc. and the search engine can use thisstereotype to find the best results.It is clear that a stereotype profile must learn over time to keepup with the interests of its users. Interests of a group of peoplecan change over time, especially when something new isintroduced for them. However, these changes will not occur asfast as and as often as the interests of a single person could. Asstated before, a lot of new learning techniques have beenimplemented and tested for personal profiles. However thesetechniques are not yet properly tested in the context ofstereotype profiles. These techniques (or a modification of atechnique) could be very useful in a stereotype profile. On theother hand, personal learning techniques might not beapplicable to stereotype profiles, this is unknown. The aim ofthe described research is to clarify this issue and to give anindication of the usefulness of personal profile techniques in astereotype profile. Such an indication will be very useful forfurther research on the subject of stereotype profiles.The main research question throughout the research will be:“Is a (slightly adapted) learning personal profile feasible as alearning stereotype profile?”The aim of the research is to find out if a personal profile hasany use when used by multiple people with somewhat the sameinterests (stereotypes). The profile used for testing is ‘slightlyadopted’ because of time, storage and testing considerations.

The profile is explained in great detail in section 2. The researchwill not be able to give a conclusive answer because there are alot of personal profiles and different settings to test with.However, the research will give an indication whether apersonal profile can be useful as stereotype profile. To answerthis question two important sub-questions are answered. Firstly:how good can a personal profile maintain the interests ofmultiple (stereotype) users? Secondly: how vulnerable is theprofile for ‘misuse’ by people who do not fit the profile. Howthese problems are addressed is described in sections 2 and 3.The results of this research are presented in section 4.3. OVERVIEW OF THE TEST SYSTEMThe aim of the research is to give an indication of the usefulnessof a personal profile for a group of people with commoninterests. To test this, a simple internet browser with a personalprofile and personal filtering rules is developed. This browser,written in Java 1.4.2, has the basic browser options (next,forward, book marking etc) and is able to display most htmlpages. The system can run on different computers so peoplecould search what they wanted whenever they wanted. Thisgives more accurate ‘results’ than to just put people behind acomputer with the task to search whether thy feel like it or not.In this way people could use the same profile at the same time.On browser-startup the single profile is loaded from a serverusing java RMI. The profile was saved back to the server whenthe browser exits. In this way all users used the same profilewithout working on the same computer.The browser can search the internet using the free available javagoogleapi. Normal search queries can be given and a set ofresults is returned by Google. This set of results is ranked usingthe profile and displayed to the user in the normal Googlelayout with the highest ranked document on top. Besides theusual Google information every result has its old Google indexand the score it received from the profile. Unfortunately thegoogleapi is restricted to give only ten results per query.3.1 The Personal ProfileThe personal profile used for this research is the one proposedby M. Boughanem and M.Tmar in [BT02]. This profile waschosen because it is a relatively ‘new’ profile and because of itsgood results and its simplicity. The latter reason was prettyimportant because of the limited time in which the research hadto be done.In principle, any personal profile could serf as a stereotypeprofile. However the results might very well differ because ofthe learning and filtering techniques used. Logically this shouldnot be the case because a good profile should know the interestsof its users no matter what techniques are used. It should beinteresting to see the differences between this research and thesame research with different personal profiles.The profile (P) as described in [BT02] consists of a collectionterm-weight pairs and is represented as follows:P = (… )In this representation t x is the xth term and w x its correspondingcurrent weight. Some extra meta-data, like the occurrence of aterm until now, and the number of documents it was found in, isassociated with every term in the profile. In our application onlythe important words of interesting documents are stored in theprofile, this in contrast with the original application of [BT02]which contains entire documents. Important words aredetermined by the layout tags in the html pages like bold, italic,links and headers. Trivial words like ‘the’, ‘and’, ‘below’ etcare filtered out using a predefined wordlist with 180 Dutch andEnglish words. The entire profile had to be learned during thetest, there were no initial terms included in the profile beforestarting the test.3.2 Filtering Documents using the ProfileThe profile is used to filter documents presented by Google.This filtering is done in the exact same way as described in[BT02]. The calculation done for each term in each document isthe following:tint in :tf i :tfi=dsnh3+ h4+ tfiAdsNlog( + 1)ntweight of term t (i th term in the n th document)frequency of term t i in document nh3, h4: constant parameters. The same values as in [BT02]h3 = 0,2 and h4=0,7.ds n : document size (number of words in document n)Ads:Average document size (average number of words)N: number of all incoming documentsn t :number of documents containing term tFrom every document the important terms, defined by htmllayout tags, are listed. The importance of each term is calculatedwith respect to the entire incoming document collection. Thismeans that the importance of the terms for their document iscalculated and how often the term occurs in other documents inthe collection. In this way terms that occur in most of theincoming documents become less important than terms that onlyoccur in one or two documents.After calculating the weight of each term the relevance value ofall incoming documents can be calculated using the profile.This calculation is done as follows:Discore = ∑ = 0tinD score :t in :i=# terms∗ pwtdocument scoreterm weight as described beforepw t : profile weight of term t.The document score is simply the sum of all the words in thedocument multiplied with the word-weight in the profile.The document with the highest score will be presented on top ofthe list, the document with the second highest score below thisone etc. During this filtering the profile is not adjusted.3.3 Learning the ProfileThe profile does not learn during filtering activities. It ishowever vital for any profile to learn and to represents itsuser(s) interests as accurate as possible. To do this, the profilemust know (or must be told) which documents are important forits user and which are not. In general there are two ways to ‘tell’the profile that a document is interesting. The first method isthrough active ranking, the user gives numbers to alldocuments. This number is an indication of the importance ofthe document and whether it should be included in the profile.The second method is by analyzing the user’s reading and inthis case browsing behavior. The profile in the described webbrowser learns from its users browsing behaviors as described

y Y.Seo and B.Zhang in [SZ00]. This research pointed out therelation between certain browsing behaviors and the importanceof the current document. From the four described parts: readingtime, bookmarking, scrolling and ‘following up hyperlinks’ ourbrowser only uses the reading time and bookmarking sincethese gave the best indication if a document is important[SZ00]. However, documents that are bookmarked by a userweigh two times heavier than documents that are included in theprofile due to reading time. This is done because bookmarkingis a very strong active indication from a user that the internetpage is very interesting. Both types of learning can occursimultaneously. It is even logical that a page that getsbookmarked already is defined as important because the userexceeded the reading-time-threshold. In this way one page canbe included in the profile several times giving the terms arelatively high weight. Since there is no ‘history of learnedpages’, a page can be learned very often if a person looks a lotat the same page.The reading time threshold depends on the number of words inthe document. If a user has a document open longer than#words/240 minutes the document is included in the profile.This is because the average reading time of a human is around250 words a minute [DBNL]. This number was lowered a bit sothat people have more time reading a page before it getslearned, making sure that only interesting pages get learned.When the browser realizes that an internet page is important forits user it will analyze the page and all the important terms willbe filtered out in the exact same way it is done in the documentfiltering. All of these words, except if they are defined in the listwith trivial words, will be included in the profile. If the profilealready contains the word the following formula will be used toupdate its weight:# tPt = Pt + * IiDsPt:#t:Ds:Ii:weight of term t in profile Pthe number that term t occurs in the documentdocument size (number of words)Importance indication, 1 for reading time and 2 forbookmarkingIf a term is not yet included in the profile the term will be addedusing the same formula as above using Pt = 0. However, thedescribed method here is not the one presented in [BT02]. Thatmethod was a bit too complicated and very hard to implement inthe short time frame I had.As described above more than one person can use the profile atthe same time. To be able to use the same profile at differentlocation the profile was stored on a server. Distributedprogramming is used to load and store the profile. Whenmultiple users use the profile at the same time it is not desirablethat an update profile is overwritten when it is saved on theserver. To avoid this, only changes in the profile made by usersare transmitted to the server. The server updates the profileusing these changes. This way all changes made by users will beequally included in the profile.The profile decreases the weights of all terms when the profileis loaded to avoid the situation where the profile continualitygrows having terms that were only once included. The lessimportant terms will become less and less important until theirweight is below a predefined threshold where the term isremoved from the profile. During the tests the decay rate was0.99 (Pt = Pt*0.99) every time the profile was loaded and thethreshold was 0.0005 (if Pt

average absolute scores from the participants and from theprofile are presented in table 1.Table 1: average absolute scores per categoryParticipants scoresProfile scoresall documents 5,09 4,53CS-documents 6,00 6,40non-CS-documents 2,94 3,13neutral-documents 5,69 4,62These results show that in general the profile gives low scoresfor all categories. There are very little 9’s and 10’s among theprofile scores. However, it is clear that CS documents get higherscores than the other documents while non-CS documents getthe lowest scores. So, after filtering the profile will present CSpages on top, neutral pages in the middle and non-CS pages atthe bottom of the list. This is in general what we want.For each category the average deviation (AD) between thescores of the profile and the (average) scores of the participantsis calculated. This AD is the difference between theparticipants score and the profile score for each document,devided by the number of documents in the category. This ADis an indication of the accuracy of the rankings from the profilefor the pages in that category. The less this AD the bettercorresponds the participants score with the system score (for acertain category). The results for the ranking method in whichevery document is ranked in context with the other documentsare presented in table 2.Table 2: results of the first testDocument categoryAverage deviationall documents 2,26CS documents 1,27non-CS documents 2,10neutral documents 2,40The deviation for all documents show that on average theprofile ranks every document 2,26 points more or less than theparticipants. This score indicates very low accuracy from theprofile for the interests of computer science students. Theprobable reason for this score is the low accuracy for neutraldocuments. These neutral documents made up the largest partfrom the test set and are pretty hard for the profile to rank. Adeeper insight in the ranking or neutral documents is describedat the bottom of this page.The results show that the ranking of CS documents is better thanthe average ranking. This seems logical since the profileprobably learned a lot of computer science pages. But still, theprofile-score differs more than one point from the rankings ofthe participants and is off by 3 points in the worst case in thetest-set. Hover the absolute scores of the CS pages are relativelyhigh which indicates that in normal searching and filtering CSpages will occur in the top of results. The results from table 2also show that the scores for CS documents are more accuratethan the scores for the other categories.The pages in the non-CS category are typically pages withcontents for very young or very old people (like pages withtoy’s or about diseases). These pages are generally ranked verylow by the profile and by the participants. However, thedeviation in this category is pretty high indicating poor accuracyfrom the profile. This means that the profile could be missrankingthe pages in this category.The results from both tables show that it is hard for the profileto rank the neutral documents. The deviation is very large andthe average score for this category is 1 point lower than thescores from the participants. The participants gave verydifferent scores to these pages because of their interest outsideCS. Some love sports while others do not care a lot about sportsand are interested in cars. Because these individual interestsdiffer a lot it is impossible for the profile to give accuratescores. Most of these pages are ranked moderate by the profilewith scores between 5 and 7. On average the filtering willpresent this kind of pages after the CS pages and before the CSpages. But because of the low accuracy for this category pagescan end up all over the ranked list.5.2 Results of the second testThe second test was the use of the profile for one week by twopeople who do not ‘fit’ the stereotype computer science student.The aim of this test was to find out how vulnerable the profile isfor ‘misuse’ by other people. The two people started with theprofile as described in section 5.1. After this week the profileranked the same document set as before. The same diagrams aspresented in section 5.1 will be used to describe the profile. Theparticipants scores will be the same as in table 1 because theserepresent the interests of ‘computer-science students’.Table 3: average absolute scores per categoryparticipantsprofileall documents 5,09 4,82CS-documents 6,00 6,13non-CS-documents 2,94 3,95neutral-documents 5,69 4,87These results show the average score of all documents did notchange a lot. On average the scores are a bit higher than in thefirst results. This is probably because of the learning of newterms that add up to the words already in the profile.The scores of some of the categories changed pretty radically.The weights in profile for CS words seem to be lowered becauseof the ‘misuse’. This is logical because every time the profile isloaded the weights of every term is lowered a bit (see section3.3). So after this week all the terms that were not learned againare lower than before, resulting in lower scores.While the CS-terms in the profile were lowered, other(presumably non CS terms) words were included in the profile.This resulted in higher scores for the non CS-documents. Thescore for these documents is still not high, but it was raised byalmost one point in one week time by only two people. Within afew weeks the profile would probably have lost the entirecomputer-science component.The average score for neutral documents does not seem to bechanged much. However, the scores for these documents haveshifted. One of the documents that got a four after the first testgot a six after the second test. For another document it was theother way around and got a lower score after the second test.This in contrast with the other categories where the scores seemto be solely lowered or raised.The results in table 4 indicate that the accuracy has not changedmuch. For some categories the deviation is higher, for others itis lower than before. The most striking result is the lowered

Table 4: results of the first testDocument categorydeviation for the CS documents. This means that the profile hasbecome more accurate in ranking CS documents. This fact alsofollows from table 3 where the profile score has become closerto the participants score for the CS category. The same goes forthe non-CS category where the deviation has increased becauseof the raise in scores.5.3 History of the profileDuring the full testing period the system maintained a log withmeta-data regarding the profile. In this log the number of termsin the profile was logged along with the terms that wereremoved from the profile because their weight dropped belowthe threshold (section 3.3). Figure 5 displays the number ofterms over time for the full testing period.1000500Average deviationall documents 2,20CS documents 1,15non-CS documents 2,56neutral documents 2,51week1 week2 week3Figure 5: number of terms in the profile over timeThe first day of the first test period no terms were included inthe profile. On the second day a great number of documentswere learned and the profile consisted of 860 words. In the restof the first week a few pages were learned, but more terms wereremoved from the profile because their weight got below thethreshold. However, hardly any CS terms was removed duringthis time. In the last two days the participants forgot theirobligation to use the system, the profile was used very little andno new terms were included in the profile. At the end of theweek the profile contained 590 terms. In the second week theparticipants were reminded of the test and the number of termsin the profile steadily increased up to 880 words. At this pointin time the first ranking of the document set took place and thesecond group of people started to (miss) use the profile. Duringthe last week of testing some 200 new terms were included inthe profile while 170 terms were removed from it. Among theseterms were some computer-science terms like ‘UML’ and‘HashSet’. However most removed terms were neutral, meaningthat most CS terms had a high enough weight to stay above thethreshold. The test period ended containing 908 terms.These results correspond with the results found in section 5.2.The CS-documents got lower scores because some of the CStermswere removed from the profile and others CS-terms got alower weight. However not a lot of CS-terms were removed andthe score did not drop dramatically. Other terms were includedin the profile increasing the scores for the non-CS and neutraldocuments.6. CONCLUSIONIt is hard to conclude anything from the results since thenumber of participants was not very high. If one of theparticipants gave an extremely high or extremely low score ithas big consequences for the score of a document. In the testsdone, one of the participants ranked a CS-document with a twowhile the others ranked it between a six and an eight loweringthe average with 1 point. However the results do indicate someconclusions.The results show a clear preference for computer-sciencedocuments. These documents get the highest score and theaccuracy for these documents is better than for the otherdocuments. This means that in general the profile will placecomputer science documents ‘above’ any other documents. Thisis a good property for a stereotype profile.Non-CS documents clearly get the lowest scores. However, theprofile is absolutely not accurate on this category making itpossible that non-CS documents get ranked pretty high. This isnot really convenient for a profile.The hardest category for the profile to learn is clearly theneutral category with documents that are interesting for somepeople and not for others. The problem is that for this categorythere is no stereotype, no ‘default’ list of terms that applies toeveryone. This means that individual interests and hobby’s cannot be learned by the profile and can get ranked very low. Sowhen someone is searching for specific (neutral) informationthe right source might be at the bottom of the ranked list! Inconventional stereotype profiles this problem also occurs, butless prominent than in the profile used for this research. In forexample collaborative filtering every user has his ‘own littleprofile’. People are compared with others using these opinionsto find out other interests of the user. Because every user has hisown little profile it is possible to learn individual interestsoutside the stereotype. This is not possible in a personal profile.The results from the second test show that it is not very hard tochange the profile. While CS terms maintain in the profile a lotof extra terms can easily be inserted making it harder for theprofile to choose between CS-documents and for exampleneutral documents. It is very likely that a few people who do notfit the profile can cause a dramatic raise in scores for the neutraldocuments.Al this indicates that a personal profile will not score very goodif it is used as a stereotype profile with a lot of users. The morepeople use the profile the more neutral terms will be learned(because of al the different interests outside the stereotype).This will probably have a negative influence on the scores forthe documents that fit the profile. However, the personal profilecould be used by people that only used it for searchingdocuments that fit a stereotype. That profile will learn the rightinterests pretty fast and has little chance of getting messed upbecause of misuse. Conclusive to say that a personal profile willnot score very well in public places with lots of people (like theearlier mentioned library). But it might do pretty good filteringdocuments for company departments.7. DISCUSSIONUnfortunately the time frame and number of participants wasnot enough to do a very thorough research on this subject. Withmore time and specially with more participants the same

esearch will probably get more reliable results to found aconclusion on. Furthermore the type of personal profile usedmight have great influence on the outcome of such a research.In this research a rather simple personal profile was used toimplement the system within the timeframe. It would beinteresting to see how other personal profiles score in somewhatthe same research as presented here.In this system only the important terms of a document werelearned if the system thought that a document was interesting.This was done to omit the implementation to select nouns froma text, again for time frame reasons. In the currentimplementation important terms are selected using certainHTML tags like links, bold and italic. Pages that are learnedthat contain hardly any of these HTML-tags obviously do nothave a lot of influence in the profile. The page might be veryinteresting for the stereotype with lots of pages but will not beranked very high in the next search. In the original personalprofile this is not the case. In the original implementation theprofile would have contained far more words than the currentimplementation has after the testing period. This might have anextreme effect on the results as they are presented in this paper.In section five it is mentioned that there are some rather bigdifferences in the rankings of the participants for the CSdocuments.This gives an indication that the participants havedifferent interests within the field of computer science. Maybethe stereotype used for this research was to ‘broad’ and a fieldwithin computer-science (like software design) would haveshown better results. As indicated above the profile willprobably score the best if it is used by a (rather small) group ofpeople with the ‘exact’ same interests.ACKNOWLEDGMENTSI would like to thank Dr. M. Poel, Dr. E.M.A.G. van Dijk andM.J. Edwards, my peers and fellow-students for theirencouragement and support. I would also want to thank theparticipants of this research for their commitment.REFERENCE[BT02][DBNL][KSS03]Boughane,m M., Tmar, M.; Incremental adaptivefiltering: profile learning and thresholdcalibration; Proceedings of the 2002 ACMsymposium on Applied computing; 2002; pp. 640-644; DOI-Bookmark:http://doi.acm.org/10.1145/508791.508915Digitale Bibliotheek der Nederlandse Letteren;Adres:http://www.dbnl.org/tekst/peer002lees01_01/peer002lees01_01_0011.htmKuflik, T., Shapira, B., Shoval, P.; Stereotype-Based versus Personal-Based Filtering Rules inInformation Filtering Systems; Journal of theAmerican Society for Information Technology,2003; pp. 243–250, DOI-Bookmark:http://www3.interscience.wiley.com/cgibin/fulltext/102523271/PDFSTART[SHY04] Sugiyama, K., Hatano,K., Yoshikawa, M.;Adaptive web search based on user profileconstructed without any effort from users;Proceedings of the 13th international conferenceon World Wide Web; 2004; pp. 675–684; DOI-Bookmark:http://doi.acm.org/10.1145/988672.988764[WIY99][SZ00]Widyantoro, D.H., Ioerger, T.R., Yen, J.; Anadaptive algorithm for learning changes in userinterests; Proceedings of the eighth internationalconference on Information and knowledgemanagement; 1999; pp. 405-412; DOI-bookmark:http://doi.acm.org/10.1145/319950.323230Seo Y., Zhang, B.; Learning user's preferences byanalyzing Web-browsing behaviours; Proceedingsof the fourth international conference onAutonomous agents; 2000; pp. 381-387; DOIbookmark:http://doi.acm.org/10.1145/336595.337546