A Wordnet from the Ground Up

3.4. Measures of Semantic Relatedness 63• text windows – the context is a window (a whole document in some specialcases); co-occurrence with particular LUs serves as constraints;• lexico-syntactic constraints – the context is a sentence, clause or phrase; lexicosyntacticrelations serve as constraints.Mohammad and Hirst (2006) write that this distinguishes between measures ofsemantic relatedness and semantic similarity, but we feel that intermediate methodsare quite conceivable. For example, one can combine lexico-syntactic constraints withco-occurrences in the description of context. So, there is a continuum of methods withthese two extremes.In the seminal paper on Latent Semantic Analysis [LSA] (Landauer and Dumais,1997), a context is simply the whole document (longer documents were truncated toa predefined size). The created co-incidence matrix (also called co-occurrence matrix)describes nouns 7 by the frequencies of their occurrences across documents. Rows correspondto nouns (60 768), columns to documents (30 473), and a cell M[n i , d j ] storesthe number of occurrences of the noun n i in the document d j . The initial cell valuesare then weighted by the logent function (Section 3.4.4) and the whole matrix is transformedby Singular Value Decomposition (SVD) (Berry, 1992) to a matrix of reduceddimensions. The SVD transformation not only improves the efficiency by reducing therow size but also – much more important – emphasises relatedness 8 between particularnouns or its absence. The final MSR value is calculated by comparing, using the cosinemeasure, rows of the reduced matrix that describe particular LUs. The relativelygood result of 64.4% achieved in the Test of English as a Foreign Language (discussedbriefly in Section 3.3) may have been due to the high quality of the corpus: GrolierEncyclopedia.In order to overcome the corpus size restriction induced in LSA by the applicationof SVD, Schütze (1998) proposed a method called Word Space. A text window movesacross documents. At each position MP of the window, statistics are collected: cooccurrenceof a word in the centre of the context with a number of meaning bearers(selected general words). Turney (2001) used the Altavista search engine to search forco-occurrences of LUs in millions of documents on the Internet and thus to calculatean MSR.Experiments performed on Polish data (Piasecki and Broda, 2007) suggest thattext-window contexts described by LU co-occurrences result in MSRs that produce7 Only noun word forms were described in the experiment of Landauer and Dumais (1997).8 Landauer and Dumais (1997) wrote about “similarity”, but in keeping with the earlier remarks weprefer to talk about semantic relatedness, because LSA is a typical text-window MSR.