User-Centered Evaluation of Information Retrieval - ideals

More documents

Recommendations

Info

100 Evaluation of Public Services ir Personnel analysis, design, and evaluation that is contextual in nature is needed; such a framework is both interpretive and phenomenological. It implies that information retrieval tasks are embedded in everyday life, and that meanings arise from individuals and from situations and are not generalizable except in a very limited sense. Users are diverse, and their situations are diverse as well. Their needs differ depending on their situation in time and space. Information systems may therefore differ, offering diverse capabilities often simultaneously within the same system which provide an array of options the user can select. For example, such systems may offer interfaces tailored to many skill and knowledge levels; they may allow users to customize their access by adding their own entry vocabularies or remembering preferred search parameters; or they may provide a variety of output and display options. In order to move beyond the present-day large, rather brittle systems which are designed to be evaluated on precision and recall, evaluation studies must be conducted that can be used in the design of new systems. By focusing on users as the basis for evaluative criteria, new systems that are more responsive and adaptive to diverse situations can be created. User-centered criteria affective measures such as user satisfaction and situational factors such as context are beginning to be used in research and evaluation. But this is just a beginning. Librarians and researchers alike must retain and refine their powers of critical observation about user behavior and attempt to look at both the antecedents and the results of information retrieval. The methods used to gain insight into these issues are frequently case studies, focus groups, or in-depth interviews which, when combined with objective measures, can afford effective methods of research and evaluation. When placing the user at the center of evaluations, it is important not to take behaviors at face value but to probe beneath the surface. In order to do this successfully, it can mean small scale, in-depth studies carried out by astute, thoughtful individuals a combination of both practitioners and researchers. ideally, REFERENCES Bawden, D. (1990). User-oriented evaluation of information systems and services. Brookfield, VT: Gower. Bates, M. J. (1984). The fallacy of the perfect thirty-item online search. RQ, 24(1), 43- 50. Cleverdon, C. W. (1962). Report on testing and analysis of an investigation into the corporate efficiency of indexing systems. Cranfield, England: College of Aeronautics. Cleverdon, C. W., & Keen, M. (1966). ASLIB Cranfield Research Project: Factors determining the performance of smaller type indexing systems: Vol. 2. Bedford, England: Cyril Cleverdon.
User-Centered Evaluation of Information Retrieval 101 Cleverdon, C. W.; Mills, J.; & Keen, M. (1966). ASLIB Cranfield Research Project: Factors determining the performance of smaller type indexing systems: Vol. 1. Design. Bedford, England: Cyril Cleverdon. Dalrymple, P. W. (1990). Retrieval by reformulation in two university library catalogs: Toward a cognitive model of searching behavior. Journal of the American Society for Information Science, 41(4), 272-281. Dalrymple, P. W., & Zweizig, D. L. (1990). Users' experience of information retrieval systems: A study of the relationship between affective measures and searching behavior. Unpublished manuscript. Dervin, B., & Nilan, M. (1986). Information needs and information uses. In M. Williams (Ed.), Annual review of information science and technology (Vol. 21, pp. 3-33). White Plains, NY: Knowledge Industry. Fenichel, C. H. (1980). Intermediary searchers' satisfaction with the results of their searches. In A. R. Benenfeld & E. J. Kazlauskas (Eds.), American Society for Information Science Proceedings, Vol. 17: Communicating information (Paper presented at the 43rd annual ASIS meeting, October 5-10, 1980) (pp. 58-60). White Plains, NY: Knowledge Industry. Fenichel, C. H. (1980-81). The process of searching online bibliographic databases: A review of research. Library Research, 2(2), 107-127. Fidel, R. (1986). Toward expert systems for the selection of search keys. Journal of the American Society for Information Science, 37(1), 37-44. Harter, S. P. (1990). Search term combinations and retrieval overlap: A proposed methodology and case study. Journal of the American Society for Information Science, 41(2), 132-146. Haynes, R. B.; McKibbon, K. A.; Walker, C. J.; Ryan, N.; Fitzgerald, D.; & Ramsden, M. F. (1990). Online access to MEDLINE in clinical settings. Annals of Internal Medicine, 772(1), 78-84. Katzer, J. (1972). The development of a semantic differential to assess users' attitudes towards an on-line interactive reference retrieval system. Journal of the American Society for Information Science, 23(2), 122-128. Katzer, J. (1987). User studies, information science, and communication. The Canadian Journal of Information Science, 72(3,4), 15-30. Kuhlthau, C. C. (1988). Developing a model of the library search process: Cognitive and affective aspects. RQ, 28(2), 232-242. Lancaster, F. W. (1977). The measurement and evaluation of library services. Washington, DC: Information Resources Press. Lancaster, F. W. (1979). Information retrieval systems: Characteristics, testing and evaluation (2nd ed.). New York: John Wiley & Sons. Markey, K. (1984). Subject searching in library catalogs: Before and after the introduction of online catalogs. Dublin, OH: OCLC. Mick, C. K.; Lindsey, G. N.; & Callahan, D. (1980). Toward usable user studies. Journal of the American Society for Information Science, 37(5), 347-356. Mischo, W. H., & Lee, J. (1987). End-user searching of bibliographic databases. In M. Williams (Ed.), Annual review of information science and technology (Vol. 22, pp. 227-63). Amsterdam: Elsevier. Nielsen, B. (1986). What they say they do and what they do: Assessing online catalog use instruction through transaction monitoring. Information Technology and Libraries, 5(1), 28-34. Norman, D. A., 8c Draper, S. W. (Eds.). (1986). User-centered system design. Hillsdale, NJ: Erlbaum. Richardson, J., Jr. (1989). Toward an expert system for reference service: A research agenda for the 1990s. College and Research Libraries, 50(2), 231-248. Sandore, B. ( 1990). Online searching: What measure satisfaction? Library and Information Science Research, 12(1), 33-54. Saracevic, T. (1976). Relevance: A review of the literature and a framework for thinking on the notion in information science. In M. J. Voigt 8c M. H. Harris (Eds.), Advances in librarianship (Vol. 6, pp. 79-138). New York: Academic Press. Saracevic, T, & Kantor, P. (1988). A study of information seeking and retrieving. Journal of the American Society for Information Science, 39(3), 177-216.
Page 1 and 2: PRUDENCE WARD DALRYMPLE Assistant P
Page 3 and 4: User-Centered Evaluation of Informa
Page 15: User-Centered Evaluation of Informa

User-Centered Evaluation of Information Retrieval - ideals

Create successful ePaper yourself

Delete template?

Save as template?