Semi Automatic Indexing State of the Art - FTP Directory Listing - Nato

6. CONCLUSIONS
This report should have made evident the feasibility of mechanization in text processing. Although quality considerations
were not discussed, it should be mentioned that most authors are satisfied by the results obtained with their methods. Thus
automation in the indexing process should go ahead, particularly as other obstacles for automation in the IR process have also
been largely overcome. These were primarily due to the lack of powerful programming languages and appropriate software
either in the retrieval process, or in data base management. However, full advantage of indexing by computer assistance can only
be taken when the textual information is in machine-readable form. As there is a trend towards automatic type-setting
techniques, this condition may be fulfilled in the near future. Other techniques to transfer textual information in
machine-readable form without re-writing and coding are already within the state-of-the-art.
Automation in storage of information can then be achieved by using fully automatic and semi-automatic indexing
techniques.
The advantages of fully automatic indexing techniques are well known:
— consistency is attained as the computer assigns index terms directly from the natural language text of the document, applying
the same algorithm for each document; (In human indexing the indexer makes a separate judgement for each document.)
— simplicity of re-indexing, which is important, because a scientific library is a living thing and classification schemes must
always change according to either the aims of the library, or developments in science;
— accuracy, which is guaranteed by the ability of the computer to select, transfer and re-arrange data reliably without making
typographical errors;
— economy, achieved by large-scale processing and computing speed;
— facility for editing.
From the quality point of view some fully automatic indexing techniques can be considered to have reached already the
same level as purely intellectual indexing, at least in a production environment (37(1970), 100(1972)].
However, automatic indexing might not be fully satisfactory if high standard indexing is required. In general, one may state
the better the index, the less intellectual effort is needed to search for information. Hence, the quality required for an index
depends strictly upon the effort an average index user is willing to spend on retrieving information.
A considerable amount of research is still required in order to have the machine do it well and efficiently especially in fully
automatic text processing such as linguistic text analysis. Most approaches apply surrogates (statistical analysis) in order to
overcome lack of knowledge in linguistics. Thus, semi-automatic computer controlled techniques are often preferred. (They do
play a useful role in linguistic research also.) Machine assisted indexing can be thought of as a simulation of a manual process
combined with some of the advantages of machine processing, such as accuracy, economy and facility of editing. For obvious
reasons semi-automatic indexing is not fully consistent and re-indexing of the entire data base is costly. Economy will be
achieved in a long-range period only since a better index can be produced which is the assumption of the success and
effectiveness of any information system.
Machine-aided indexing can also take full advantage of the preferences of intellectual indexing. Indexers (cited from
[103(1969)]:
— are able to make discriminations as to the relative importance of technical concepts as they appear in an abstract or
document,
— have access to the entire document and
— can go beyond the document itself to reference books, to consultation with experts, or other sources as deemed appropriate,
to aid in properly indexing the document at hand,
— can apply inductive reasoning to formulate and index concepts which are implied by the document but not expressly stated
(assignment indexing),
— become familiar with the requirements of the users of the system by participating in search request analysis, search strategy
formulation and search screening.
Thus, semi-automatic indexing will be preferable to purely intellectual indexing. On the other hand it is believed that fully
automatic indexing techniques can be developed in the near future to satisfy requirements. Machine-assisted methods may help
in achieving this aim.
Acknowledgements
/ wish to express my gratitude to Miss G. Pozzi. Head of the European Scientific Information Processing Centre. C.E.T.I.S., of the Commission of
the European Community, who gave her support to the compilation of this report.
In particular, I am grateful to Prof. F. W. Lancaster and my colleague W. Kolar for fruitful discussions and some useful suggestions.
IMSI but not least, I acknowledge the helpful assistance that I received from the Library staff, the Publication and Typing Office of the EURATOM-
Joint Research Center in Ispra.