7 - Indira Gandhi Centre for Atomic Research

• Any initial letter that precedes the name, it will be transferred to the end of the name
and a comma inserted between the initials and the name
All the words / phrases are now alphabetized.
3.2 Identification of Keyword / Key phrases:
In OPACs and similar databases the elements representing the subject of the resource
usually take the form of a set of data fields that may include keywords, descriptors, subject
headings, abstract, classification codes, etc. However, in automatic extraction of keywords
from a document it is necessary to look for appropriate lexical clues. The major type of
lexical clue to the subject of a document is the set of domain terms the document contains.
This usually takes the form of keywords. The experiment reported in this paper involves the
use of a set of simple heuristics to identify keywords and key phrases in HTML documents.
The module involves the following major inputs:
• One or more HTML files constituting the documentary information resources in a
domain from which keywords / key phrases are to be extracted automatically; The
program requires that all the HTML files be in a single folder
• A database which is in effect a list of domain terms in the subject area / discipline of the
HTML files: In this study the ASIS thesaurus was employed
• A database of ‘stop words’ consisting of all non-noun words taken from the Pocket
English dictionary which itself is derived from the New Oxford Dictionary of
English.
The principal output of the program is a HTML page consisting of extracted keywords / key
phrases with hyperlinks to the HTML pages from which they were extracted.
Keyword Extraction: The major problems involved in KWE are extraction of keywords and
omission of non-significant words. The experience with techniques such as those adopted
by the popular search engines clearly brings out the need for a different approach. In this
study it was decided to experiment with a validation process using two databases of terms to
assist in the identification of keywords and non-significant words in the input file. The
validation process employed made certain assumptions:
• It was assumed that a word / phrase in the input HTML file that is also part of a
controlled vocabulary in the concerned subject domain is a key word / key phrase
with a high probability of indicating the subject content of the input file.
• Non-noun words in the input file are assumed to be non-significant words.
• In the present experiment the following inputs / tools were employed:
• A paper entitled ‘Information Retrieval and Cognitive Research’ was used as the
input HTML document to test the utility and limitations of the Program. An idea of
the paper can be had from the details given in the Table 1 below.
• As for identifying keywords and key phrases in the input file online tools were used..
• The ASIS thesaurus (http://www.asis.org/Publications/Thesaurus/isframe.htm)
i. Stop-word Terms (ST): Uncontrolled vocabularies have always presented
problems in IR. The most common words in English may account for 50% or
more of any given text. Their semantic content measured in terms of their
value in describing / indicating the subject matter of the text is minimal.
Further, such words tend to lessen the impact of frequency differences among
the less common words. In addition, they necessitate a large amount of
unnecessary processing. In all methods of automatic indexing such less
significant words are ignored based on a stop-word list of such words. As
already mentioned in the present experiment all non-noun words were
63