7 - Indira Gandhi Centre for Atomic Research

• Proper names and
• Keywords / key phrases from web documents
2. Scope and Objectives:
The important elements of description (metadata) of any documentary resource from the
point of view of their utility as search keys are:
• The proper names associated with the resource; This would include names of
persons, names of institutions and other corporate bodies; and
• Keywords and Key phrases representing the subject content of the document.
If effective mechanisms could be developed and implemented to identify these and extract
them from electronic resources these will be useful metadata. The experiments reported in
this paper were carried out to develop and test algorithms to identify and extract Proper
names and keywords from web documents. The mechanism uses reasonably fast and robust
heuristics to identify proper names, keywords and extract them from the web resources.
3. The experiment:
There are basically two aspects to this study. The first one focused on identification and
extraction of proper names and the second one on keywords. A corpus of HTML documents
on the Web was used as the test bed to experiment with the algorithms developed. The first
step was to arrive at a corpus of electronic texts to experiment with. In planning such a study
it was decided, keeping in mind a wide variety of factors, to conduct the experiments with
documents falling in a subject domain. This was necessary as one component of the study
focused on extraction of keywords.
3.1 Identification of Proper names: For the purpose of this experiment and study ‘name
extraction’ is defined as the process of identifying and extracting personal names from
unstructured web texts in the English language. A set of rules to enable a computer to
identify names in all their variations is an essential component of the kind of text processing
application described in this paper. Literature on the subject has references to a few rules
identified and applied to extract names. Most of these rules are derived from standard
conventions that are widely employed by authors of properly edited texts in English. Some
of the key indicators that were used included:
‣ Use of legend words such as Mr. Miss, Ms. etc is a good indicator that the following
word / words denotes a name;
‣ Use of Corporate trigger words such as ‘University’, ‘College’, ‘School’, etc., is a
good indicator to identify the organization names. Prepositions are widely used in
corporate names to link legend words with other words (e.g. University of
California); this has been exploited to identify complete corporate names
‣ Initial capitalization is also an indicator of names since the convention of English
language requires that each word of a name start with an uppercase letter. In this
module, therefore, identification of initial capitalization has been used in formulating
the algorithm.
‣ Example: Powell
‣ First Name, Middle Name and Last Name starting with uppercase letters is also a
good indicator for names 2 .
‣ Example: Mohandas Karamchand Gandhi
61