Web Mining and Social Networking: Techniques and ... - tud.ttu.ee

78 4 Web Content MiningInput queryOnline stepOffline stepSearch Engine qinput, term1,,termxSimilarity computation SuggestionqaqbqcPast Web queriesSearch Engineqa, term1,,termxqb, term1,,termx Fig. 4.1. Framework of the traditional pseudo relevance based suggestion using the term featurespaceof Web users, but it has a critical drawback. If the input query by a user is totally new to currentWeb logs, its click information is not accessible from the Web query logs for query similaritycomputation. In this case, pseudo relevance can be used to expand the expressions of queriesby features extracted from the top search results returned by search engines. Two traditionalfeature spaces are terms and URLs of these search results.Suppose that we have already obtained a set of candidate queries which are past queriessubmitted by previous users. The task of query suggestion is to suggest the most related candidatesfor an target input query. In Figure 1 we illustrate the overview of the traditional pseudorelevance based suggestion which uses the term feature space.In the offline step, the top search results of past queries are retrieved by a search engine,and terms are extracted from the search result pages. In this manner, each past query is representedby the terms of its search result pages. In the online step, the same process is used toget the term expression of an input query. The query to query similarity is computed based ontheir term expressions. The past queries with the highest similarity scores are recommended tousers. Pseudo relevance based suggestion using the URL feature space works in a similar wayshown in Figure 1. The only difference is placing the terms with URLs in query expression.If we use terms of search results as features. We can obtain the following vector expression:V(q i )={t 1 ,t 2 ,···,t k }. The similarity function can be the well-known cosine coefficient,defined asSim(q i ,q j )= V(q i) ·V (q j )|V(q i )||V(q j )| . (4.5)URLs can also be used to express queries and compute query-to-query similarity. Let q iand q j be two queries, and U(q i ) and U(q j ) be the two sets of URLs returned by a searchengine (here i.e., Google) in response to the two queries q i and q j respectively. In this case,Jaccard is an intuitive and suitable similarity measure, defined as:Jaccard(q i ,q j )= |U(q i) ∩U(q j )||U(q i ) ∪U(q j )| . (4.6)The value of the defined similarity between two queries lies in the range [0, 1]: 1 if they areexactly the same URLs, and 0 if they have no URLs in common.After the feature enrichment of a query, we can apply traditional text processing approacheson the enriched queries,such as text classification, text clustering, topic detectionand so forth.