A Review of the Literature on Assessment in Academic and ...

REPORTS FROM THE FIELD 

A <strong>Review</strong> <strong>of</strong> <strong>the</strong> <strong>Literature</strong> on Assessment 

in Academic and Research Libraries, 

2005 to August 2011 

Jon R. Hufford 

Introduction 

Assessment in United States higher education has become very important in recent 

years. Virtually all colleges and universities are now striving to prove through empirical 

evidence that <strong>the</strong>y are committed to improving student learning. Created in late 2005, 

<strong>the</strong> US Department <strong>of</strong> Education’s Commission on <strong>the</strong> Future <strong>of</strong> Higher Education was 

charged with <strong>the</strong> task <strong>of</strong> “developing a strategy for higher education to meet <strong>the</strong> needs 

<strong>of</strong> America’s population and address <strong>the</strong> economic and workforce needs <strong>of</strong> <strong>the</strong> future.” 1 

The Commission’s final report, A Test <strong>of</strong> Leadership: Charting <strong>the</strong> Future <strong>of</strong> United States 

Higher Education, makes several recommendations for reform and states in a summary 

that “improved accountability is vital to ensuring <strong>the</strong> success <strong>of</strong> all <strong>the</strong> o<strong>the</strong>r reforms 

we propose. Colleges and universities must become more transparent about cost, price, 

and student success outcomes, and must willingly share this information with students 

and families.” Though <strong>the</strong> report was not an early wake-up call for assessment, since 

articles on <strong>the</strong> need and importance <strong>of</strong> assessment in higher education had been published 

in <strong>the</strong> literature before 2006, 2 it was an important document on <strong>the</strong> topic from a 

political and administrative perspective and has had a significant impact on campuses 

across <strong>the</strong> nation. This is because, in response to <strong>the</strong> Commission’s report, <strong>the</strong> regional 

accreditation organizations made changes in <strong>the</strong>ir standards, and <strong>the</strong>se changes have 

been directly responsible for <strong>the</strong> trend toward assessment. As might be expected, some 

<strong>of</strong> <strong>the</strong> standards <strong>of</strong> several <strong>of</strong> <strong>the</strong>se regional organizations relate to academic libraries 

and have changed <strong>the</strong> way <strong>the</strong>y are assessed. 

This article reviews <strong>the</strong> literature on assessment in academic and research libraries 

that has been published since <strong>the</strong> Commission on <strong>the</strong> Future <strong>of</strong> Higher Education was 

created in 2005 and up until August 2011. Much has been written about assessment 

This mss. is peer reviewed, copy edited, and accepted for publication, portal 13.1. 

portal: Libraries and <strong>the</strong> Academy, Vol. 13, No. 1 (2013), pp. 5–35. 

Copyright © 2013 by The Johns Hopkins University Press, Baltimore, MD 21218.

6 


over <strong>the</strong> years. The author felt that coverage <strong>of</strong> <strong>the</strong> literature <strong>of</strong> <strong>the</strong> most recent six years 

would include sources that would still be reasonably relevant for academic pr<strong>of</strong>essionals 

in 2011. The author was also interested in providing an examination <strong>of</strong> <strong>the</strong> coverage 

since <strong>the</strong> Commission’s report to disclose new developments, ideas, and directions in 

<strong>the</strong> assessment <strong>of</strong> academic and research libraries. Also, to <strong>of</strong>fer a competent and manageable 

review <strong>of</strong> <strong>the</strong> literature, <strong>the</strong> article focuses on assessments concerned with <strong>the</strong> 

management or administration <strong>of</strong> libraries and <strong>the</strong>ir collections, public services, and 

issues or projects affecting more than one department within a library. Assessments in 

<strong>the</strong> acquisitions and technical services areas have been excluded. 

The author’s goal was to treat <strong>the</strong> topic comprehensively. However, some sources 

were not considered significant enough to be included, and it is possible that o<strong>the</strong>r 

sources that perhaps should have been included were inadvertently left out. A representative 

selection <strong>of</strong> noteworthy older monographs and textbooks generally recognized 

for <strong>the</strong>ir contribution to academic and research library assessment has been included, 

along with a selection <strong>of</strong> publications on assessment in higher education that provides 

a more comprehensive perspective <strong>of</strong> <strong>the</strong> topic. Though <strong>the</strong> majority <strong>of</strong> <strong>the</strong> sources 

reviewed in this article were published in <strong>the</strong> United States and cover developments in 

<strong>the</strong> United States, some sources published in international or foreign publications that 

cover developments in <strong>the</strong> United Kingdom, Canada, Australia, or South Africa were 

included because <strong>the</strong>y discuss developments similar to those taking place in American 

academic and research libraries and provide important insights into <strong>the</strong> topic. 

Assessment is <strong>of</strong>ten equated with evaluation. To education pr<strong>of</strong>essionals, assessment 

occurs when a researcher wants to discover what a student knows or can do, 

while evaluation is used to determine <strong>the</strong> value <strong>of</strong> a course or program. Authors <strong>of</strong> 

books focused on academic and research libraries normally use evaluation when <strong>the</strong>y 

discuss <strong>the</strong> measurement (or evaluation) <strong>of</strong> a library’s operations and services. In The 

Measurement and Evaluation <strong>of</strong> Library Services, F.W. Lancaster explains in his definition <strong>of</strong> 

evaluation that it “consists <strong>of</strong> <strong>the</strong> comparison <strong>of</strong> performance with <strong>the</strong> objectives <strong>of</strong> <strong>the</strong> 

agency, in order to determine (a) whe<strong>the</strong>r <strong>the</strong>re has been any change in performance for 

a given time period, (b) if so, whe<strong>the</strong>r <strong>the</strong> change is in <strong>the</strong> desired direction, and (c), if 

so, to what extent.” 3 On <strong>the</strong> o<strong>the</strong>r hand, in <strong>the</strong> Association <strong>of</strong> Research Libraries’ SPEC 

Kit 303: Library Assessment, Stephanie Wright and Lynda White state that “to assess, 

in general, is to determine <strong>the</strong> importance, size, or value <strong>of</strong>; to evaluate. Library staff 

assess operations by collecting, interpreting, and using data to make decisions and to 

improve customer service. They study internal processes, levels and quality <strong>of</strong> service, 

and library impact on institutional goals.” 4 These two quotations, both from reputable 

publications in <strong>the</strong> library pr<strong>of</strong>ession, indicate that <strong>the</strong> use <strong>of</strong> <strong>the</strong> words assessment and 

evaluation among librarians is sometimes ambiguous, or has changed over <strong>the</strong> years. 

Given <strong>the</strong> ambiguity, <strong>the</strong> author <strong>of</strong> this article decided to use in his analyses <strong>of</strong> sources 

<strong>the</strong> word that is found in <strong>the</strong> source being described. This explains why assessment 

is used in some paragraphs and evaluation in o<strong>the</strong>rs. When not analyzing particular 

sources, assessment is generally used and is meant to embrace learning assessment as 

well as evaluation (or assessment) <strong>of</strong> academic or research library operations and services. 

The author searched several databases using <strong>the</strong> keywords “academic libraries” 

or “research libraries” and “assessment” or “evaluation.” User surveys employed in 

This mss. is peer reviewed, copy edited, and accepted for publication, portal 13.1.

Jon R. Hufford 7 

library and information science ascertain <strong>the</strong> characteristics <strong>of</strong> information users and 

<strong>the</strong>ir behavior. As a category, <strong>the</strong>y include more than studies that concentrate on assessing 

or evaluating. Materials on user studies are included in this article if <strong>the</strong>ir purpose 

was to assess. The databases searched included Web <strong>of</strong> Science; <strong>the</strong> FirstSearch databases 

ArticleFirst, Dissertations, Ebooks, ECO, ERIC, PapersFirst, Proceedings, Periodical Abstracts, 

WilsonSelectPlus, and WorldCat; <strong>the</strong> Wilson databases Library <strong>Literature</strong> and Information 

Science and OmniFile Full Text Mega; and <strong>the</strong> EBSCO databases Academic Search Complete, 

Education Research Complete, and Library, Information Science, and Technology Abstracts. 

Several types <strong>of</strong> publications were examined for inclusion in <strong>the</strong> article, including articles, 

conference papers, monographs, textbooks, reports, websites, and dissertations. 

The sources reviewed are organized under broad topics, and at <strong>the</strong> beginning <strong>of</strong> 

each <strong>of</strong> <strong>the</strong>se topical groupings <strong>the</strong> author has identified those sources that are more 

significant than <strong>the</strong> rest. In a 

couple <strong>of</strong> cases papers from The sources reviewed are organized under 

<strong>the</strong> proceedings <strong>of</strong> a conference 

are treated toge<strong>the</strong>r 

broad topics, and at <strong>the</strong> beginning <strong>of</strong> each 

even though <strong>the</strong>y are on 

different topics. Within each 

topic <strong>the</strong> sources are grouped 

by type <strong>of</strong> publication. They 

are organized within <strong>the</strong>se 

groups by date <strong>of</strong> publication in reverse chronological order. The subject content <strong>of</strong> several 

<strong>of</strong> <strong>the</strong> sources is not conducive to arrangement in categories. The author <strong>the</strong>refore 

decided not to use topical subdivisions within <strong>the</strong> broad topics. 

Some Noteworthy Works Published Before 2005 

<strong>of</strong> <strong>the</strong>se topical groupings <strong>the</strong> author has 

identified those sources that are more significant 

than <strong>the</strong> rest. 

In previous decades input and output assessment played <strong>the</strong> major role in evaluating 

academic and research libraries. Data representing an institution’s input (for example, 

number <strong>of</strong> books purchased, journals subscribed to, and tables and chairs available in 

study spaces) and output (such as, number <strong>of</strong> books circulated and interlibrary loan requests 

processed) was an important component in an annual report. However, in recent 

years impact or outcomes assessment, in particular student learning-outcomes assessment, 

has played a much more important role in library assessment. This is reflected in 

<strong>the</strong> fact that much <strong>of</strong> <strong>the</strong> literature on library assessment over <strong>the</strong> past ten years or more 

is about measuring service quality using instruments such as LIBQUAL+ TM and student 

learning outcomes in information literacy classes and sessions. 

Though this change in emphasis has taken place, input and output measurement is 

still performed. A significant number <strong>of</strong> academic and research libraries submit annual 

reports to <strong>the</strong> entities <strong>the</strong>y report to, and <strong>the</strong>se reports still include input and output data. 

For this reason <strong>the</strong> author believes it is worthwhile pointing out a handful <strong>of</strong> significant 

books published before 2005 that have gained notoriety and that provide insight into 

<strong>the</strong> past status <strong>of</strong> library assessment. Several are textbooks examining research methods 

that <strong>the</strong> authors deemed useful for students in library schools, but also for practicing 

librarians. It may be argued that much <strong>of</strong> <strong>the</strong> basic methodology <strong>the</strong>y cover and <strong>the</strong> 


8 


terminology used are still relevant or may again become relevant at some time in <strong>the</strong> 

future. The content <strong>of</strong> most <strong>of</strong> <strong>the</strong>se books is applicable to all types <strong>of</strong> libraries, though 

one covers <strong>the</strong> measurement <strong>of</strong> just academic libraries. 

F. Wilfrid Lancaster is responsible for perhaps <strong>the</strong> most recognized <strong>of</strong> <strong>the</strong>se earlier 

publications. With <strong>the</strong> assistance <strong>of</strong> Mary Jane Jancich, Lancaster authored The Measurement 

and Evaluation <strong>of</strong> Library Services in 1977. 5 He and Sharon Baker authored a second 

edition in 1991. 6 The first edition received <strong>the</strong> American Library Association’s Ralph 

Shaw Award in 1978. Both editions <strong>of</strong> <strong>the</strong> work were intended primarily as a survey 

and syn<strong>the</strong>sis <strong>of</strong> <strong>the</strong> literature on evaluation, though <strong>the</strong> second edition was also meant 

to serve as a practical guide to <strong>the</strong> evaluation <strong>of</strong> libraries, including <strong>the</strong> services <strong>the</strong>y 

<strong>of</strong>fer <strong>the</strong>ir users. Also, <strong>the</strong> second edition does not limit coverage to just studies done at 

academic libraries as <strong>the</strong> first edition did, but covers studies done at public, school, and 

special libraries as well. Chapters in both editions examine collection evaluation from 

both <strong>the</strong> materials perspective and <strong>the</strong> user’s perspective, <strong>the</strong> evaluation <strong>of</strong> in-house 

use, <strong>the</strong> availability <strong>of</strong> materials, catalog use, and <strong>the</strong> reference services involved in 

answering questions and searching databases. 

Lancaster also authored a work titled If You Want to Evaluate Your Library … . A 

first edition was published in 1988, and ano<strong>the</strong>r in 1993. 7 The 1988 edition received <strong>the</strong> 

American Library Association’s G.K. Hall Award in 1989. This work complemented ra<strong>the</strong>r 

than duplicated <strong>the</strong> content <strong>of</strong> The Measurement and Evaluation <strong>of</strong> Library Services. Both 

editions were meant to be used as textbooks for teaching <strong>the</strong> subject in library schools 

and as a practical guide for practicing librarians in <strong>the</strong> selection <strong>of</strong> evaluation procedures 

to use at <strong>the</strong>ir libraries. The focus is on public services. While <strong>the</strong>y review <strong>the</strong> literature 

on library evaluation <strong>the</strong>y do not do so in a comprehensive manner. The second edition 

contains an additional chapter on <strong>the</strong> evaluation <strong>of</strong> bibliographic instruction and ano<strong>the</strong>r 

on continuous quality control applied to library services. Also, evaluations using “costeffectiveness, 

cost-benefit, and resource-sharing” methods are treated more thoroughly 

than in <strong>the</strong> first edition. 8 

Peter Hernon and Charles McClure had already published books on <strong>the</strong> evaluation 

<strong>of</strong> library services prior to <strong>the</strong> publication <strong>of</strong> Evaluation and Library Decision Making in 

1990. However, this work treats <strong>the</strong> subject <strong>of</strong> library assessment more comprehensively 

than any <strong>of</strong> <strong>the</strong>ir earlier publications. What’s more, Hernon and McClure maintain in 

<strong>the</strong>ir preface to this book that, since no previously published textbook focused on <strong>the</strong> 

entire evaluation process from <strong>the</strong> perspective <strong>of</strong> libraries, <strong>the</strong>ir book filled a void. The 

authors’ purpose in writing this book was “to assist librarians, library school students, 

and public <strong>of</strong>ficials in understanding evaluation as a research activity and <strong>the</strong> relationship 

between evaluation and planning.” 9 Among o<strong>the</strong>r things, <strong>the</strong> book reviews <strong>the</strong> literature 

on evaluation, <strong>the</strong> steps involved in conducting a study, performance measures, reporting 

a study’s findings, and possible political barriers to <strong>the</strong> implementation <strong>of</strong> evaluation 

research. Two chapters (10 and 11) include examples <strong>of</strong> evaluation studies undertaken 

at academic libraries. Evaluation and Library Decision Making complements two earlier 

publications in <strong>the</strong> Ablex Publishing Corporation’s series “Information Management, 

Policy, and Services” that Hernon co-authored with o<strong>the</strong>rs—Statistics for Library Decision 

Making and Microcomputer S<strong>of</strong>tware for Performing Statistical Analysis. The former 

discusses <strong>the</strong> application <strong>of</strong> descriptive and inferential statistics and <strong>the</strong> latter explains 

how librarians can perform statistical analyses on microcomputers. 10 



The American Library Association was responsible for <strong>the</strong> publication in 1990 <strong>of</strong> 

ano<strong>the</strong>r <strong>of</strong> <strong>the</strong>se noteworthy works published before 2005, entitled Measuring Academic 

Library Performance: a Practical Approach. Authors Nancy Van House, Beth Weil, and 

Charles McClure worked under <strong>the</strong> guidance <strong>of</strong> <strong>the</strong> Association <strong>of</strong> College and Research 

Libraries’ Ad Hoc Committee on Performance Measures. The book was designed to 

serve as a practical manual <strong>of</strong> output measures useful in evaluating <strong>the</strong> effectiveness <strong>of</strong> 

a library’s performance. It covers measures relevant only to public services. The authors 

maintain in <strong>the</strong>ir preface that <strong>the</strong> measures could be used in all types and sizes <strong>of</strong> academic 

and research libraries, are easy to apply and use, inexpensive to administer, support 

decision-making, are user oriented, and reflect common library goals and objectives. 

The authors’ purpose in writing <strong>the</strong> manual was “to promote <strong>the</strong> use <strong>of</strong> measurement 

for management decision-making in academic and research libraries.” 11 Measurements 

for classroom instruction were not included. 

Assessment in Higher Education 

A number <strong>of</strong> publications on assessment in higher education published since 2005 have 

significantly influenced <strong>the</strong> way academic and research libraries are now being assessed. 

Student learning outcomes assessment gets substantial coverage in <strong>the</strong>se publications. 

Perhaps <strong>the</strong> most important <strong>of</strong> <strong>the</strong>m is <strong>the</strong> report issued in 2006 by <strong>the</strong> Commission 

on <strong>the</strong> Future <strong>of</strong> Higher Education that is discussed in <strong>the</strong> introduction to this article. 

Ano<strong>the</strong>r significant and influential report was issued by <strong>the</strong> Council <strong>of</strong> Higher Education 

Management Associations (CHEMA) and <strong>the</strong> EDUCAUSE Center for Applied 

Research in 2006. It discloses <strong>the</strong> results <strong>of</strong> a Delphi study that used a survey distributed 

to <strong>the</strong> members <strong>of</strong> <strong>the</strong> boards <strong>of</strong> directors <strong>of</strong> each member association and interviews 

conducted with representatives <strong>of</strong> each association. CHEMA’s purpose in undertaking 

this study was to identify forces for change and to better understand <strong>the</strong>ir implications 

for higher education. CHEMA intended to add by way <strong>of</strong> <strong>the</strong> report <strong>the</strong> “voice <strong>of</strong> higher 

education’s administrative leadership to those who are seeking to understand and shape 

<strong>the</strong> future <strong>of</strong> higher education.” The report concluded that “heightened competition, 

changing revenue streams, demographics, technology, and altered public perceptions 

are creating serious threats and opportunities for higher education.” It also played a role 

in nurturing <strong>the</strong> movement in support <strong>of</strong> outcomes assessment. 12 

Revisiting Outcomes Assessment in Higher Education, published in 2006 and authored by 

Peter Hernon, Robert Dugan, and Candy Schwartz, complements Hernon and Dugan’s 

earlier publication Outcomes Assessment in Higher Education published in 2004. 13 Both include 

contributions from various groups and individuals involved in higher education, 

including members <strong>of</strong> accreditation organizations, an academic vice president, deans, 

a higher education consultant, faculty members, and librarians. Each contributor sheds 

light on how his segment <strong>of</strong> higher education views and facilitates outcomes assessment. 

A resource important because it <strong>of</strong>fers up-to-date information on outcomes assessment in 

higher education is <strong>the</strong> Council for <strong>the</strong> Advancement <strong>of</strong> Standards in Higher Education 

(CAS)’s website. The Council is a “consortium <strong>of</strong> pr<strong>of</strong>essional associations concerned 

with <strong>the</strong> development and promulgation <strong>of</strong> pr<strong>of</strong>essional standards and guidelines for 

student support programs and services in institutions <strong>of</strong> higher education.” 14 


10 


Assessing Academic and Research Libraries 

The sources discussed in this section that 

made significant contributions to <strong>the</strong> 

literature on <strong>the</strong> assessment <strong>of</strong> academic 

and research libraries include <strong>the</strong> Association 

<strong>of</strong> College and Research Libraries’ The 

Value <strong>of</strong> Academic Libraries: A comprehensive 

Research <strong>Review</strong> and Report, <strong>the</strong> Association 

<strong>of</strong> Research Libraries’ SPEC Kit 303: 

Library Assessment, <strong>the</strong> Council on Library 

and Information Resources’ No Brief 

Candle: Reconceiving Research Libraries for 

<strong>the</strong> 21 st Century, and <strong>the</strong> article authored 

by Steve Hiller, Martha Kyrillidou, and Jim 

Self that reports on assessment in North 

American research libraries. 

The sources discussed in this section that made significant contributions to <strong>the</strong> literature 

on <strong>the</strong> assessment <strong>of</strong> academic and research libraries include <strong>the</strong> Association <strong>of</strong> College 

and Research Libraries’ The Value <strong>of</strong> Academic Libraries: A comprehensive Research <strong>Review</strong> 

and Report, <strong>the</strong> Association <strong>of</strong> Research Libraries’ SPEC Kit 303: Library Assessment, <strong>the</strong> 

Council on Library and Information Resources’ No Brief Candle: Reconceiving Research 

Libraries for <strong>the</strong> 21 st Century, and <strong>the</strong> article authored by Steve Hiller, Martha Kyrillidou, 

and Jim Self that reports on assessment in North American research libraries. The 

papers presented at <strong>the</strong> Library Assessment Conference held in Baltimore in 2010 and 

subsequently published in a special issue <strong>of</strong> Library Quarterly are also worth reading, in 

particular <strong>the</strong> one by Stephen Town. 

In 2010, <strong>the</strong> Association <strong>of</strong> College and Research Libraries (ACRL) inaugurated its 

Value <strong>of</strong> Academic Libraries Initiative to assist academic librarians in participating in 

<strong>the</strong> conversations taking place on <strong>the</strong> campuses <strong>of</strong> universities and colleges and in government 

circles about assessment, accountability, and value and to identify resources to 

support <strong>the</strong>m in demonstrating 

<strong>the</strong> value <strong>of</strong> academic libraries. 

The Value <strong>of</strong> Academic Libraries: 

A Comprehensive Research <strong>Review</strong> 

and Report, researched by 

Megan Oakleaf and published 

online in September 2010 by 

ACRL, provides librarians a 

review <strong>of</strong> <strong>the</strong> literature on <strong>the</strong> 

value <strong>of</strong> libraries in an institutional 

context, suggestions for 

steps to take to demonstrate an 

academic library’s value, and a 

research agenda for <strong>the</strong> topic. 

Fur<strong>the</strong>rmore, this important 

resource reviews several assessment 

“perspectives,” including 

student learning outcomes assessment. 

It encourages librarians 

to adapt <strong>the</strong>se perspectives 

to <strong>the</strong>ir library’s circumstances. 

Ano<strong>the</strong>r product <strong>of</strong> <strong>the</strong> Initiative 

is a toolkit developed by 

<strong>the</strong> ACRL Assessment Committee that provides academic librarians with tools and 

examples to use. ACRL also plans to create more opportunities for academic librarians 

to develop assessment and research skills and to secure funds to implement <strong>the</strong> research 

agenda discussed in <strong>the</strong> report. 15 

In <strong>the</strong>ir book Surviving <strong>the</strong> Future: Academic Libraries, Quality, and Assessment, Gail 

Munde and Kenneth Marks provide information that academic librarians can use to 



improve <strong>the</strong> quality <strong>of</strong> <strong>the</strong> services at <strong>the</strong>ir institutions. 16 Their goal in writing this book 

was to provide a “thorough, logical, and integrated view <strong>of</strong> academic library assessment, 

evaluation, and quality improvement.” Ano<strong>the</strong>r book edited by Janet Hurlbert 

and published in 2008 focuses on providing assistance to college library managers in 

planning for <strong>the</strong> many directions <strong>the</strong>y must consider and act upon to keep <strong>the</strong>ir libraries 

viable. 17 Several authors contributed to <strong>the</strong> book’s content. Some <strong>of</strong> <strong>the</strong> chapters include 

studies and models that can be replicated. The volume is meant to be used as a guide 

for strategic planning. 

Joseph Mat<strong>the</strong>ws authored two monographs on academic library assessment in 

2007. Library Assessment in Higher Education provides information on research and <strong>the</strong> 

best assessment practices from a university’s perspective as well as an academic library’s 

perspective. 18 It also provides a set <strong>of</strong> 

assessment tools useful for academic 

Mat<strong>the</strong>ws argues in this volume for 

librarians generally. On <strong>the</strong> o<strong>the</strong>r hand, 

his o<strong>the</strong>r monograph—The Evaluation <strong>the</strong> adoption <strong>of</strong> broader and more 

and Measurement <strong>of</strong> Library Services—was rigorous evaluation techniques, 

written for library administrators <strong>of</strong> all 

including traditional internal measurements 

as well as more customeroriented 

measurements. 

types <strong>of</strong> libraries interested in assessing 

library services. It was also intended 

to be used by library school students. 

Mat<strong>the</strong>ws argues in this volume for <strong>the</strong> 

adoption <strong>of</strong> broader and more rigorous 

evaluation techniques, including traditional internal measurements as well as more 

customer-oriented measurements. In addition, <strong>the</strong> Association <strong>of</strong> Research Libraries’ 

SPEC Kit 303: Library Assessment published in 2007 provides both an overview <strong>of</strong> how 

library assessment activities have been developed and implemented in recent years and 

assistance in assessing where a library is in relation to its peer institutions and identifying 

best practices to help build or expand programs. 19 

The Council on Library and Information Resources’ publication No Brief Candle: 

Reconceiving Research Libraries for <strong>the</strong> 21 st Century includes eight essays, presented at a 

symposium held in Washington, DC, and several recommendations. 20 In its general 

summary <strong>of</strong> <strong>the</strong> essays and <strong>the</strong> discussions held at <strong>the</strong> symposium, <strong>the</strong> report posits <strong>the</strong> 

point <strong>of</strong> view that librarians must exercise “aggressive intervention” to better manage <strong>the</strong> 

challenges research libraries are facing. It demands change and <strong>of</strong>fers a broad research 

agenda that entails, among many o<strong>the</strong>r things, assessment at all levels including student 

learning outcomes assessment. 

Ano<strong>the</strong>r response to <strong>the</strong> campus conversations on assessment, accountability, and 

value materialized as <strong>the</strong> Library Assessment Conference held in Baltimore in October, 

2010, and sponsored by <strong>the</strong> Association <strong>of</strong> Research Libraries, <strong>the</strong> University <strong>of</strong> Virginia 

Library, and <strong>the</strong> University <strong>of</strong> Washington Libraries. The five keynote speakers at <strong>the</strong> 

Conference prepared papers that were published in a special issue <strong>of</strong> Library Quarterly in 

January 2011. In <strong>the</strong> first <strong>of</strong> <strong>the</strong>se articles, Fred Heath reviews <strong>the</strong> demands for accountability 

and <strong>the</strong> progress made in recent years. His focus is on <strong>the</strong> work <strong>the</strong> Association 

<strong>of</strong> Research Libraries and o<strong>the</strong>rs have accomplished in support <strong>of</strong> library assessment. 21 

Danuta Nitecki’s contribution <strong>of</strong>fers a framework for considering different factors affect- 


12 


ing library space assessment and insight into <strong>the</strong> relationship <strong>of</strong> space to an academic 

library’s purpose and goals. She discusses paths available in approaching assessment. 

They include considering <strong>the</strong> assessment’s purpose, <strong>the</strong> questions posed by it, data 

ga<strong>the</strong>ring methodologies, and strategies for reporting <strong>the</strong> results. 22 

Megan Oakleaf explains in her article in this special issue <strong>of</strong> Library Quarterly that 

libraries are moving beyond <strong>the</strong> assessment <strong>of</strong> individual learning outcomes to larger 

scale value assessment that measures a library’s impact on its university or college. 

She examines questions relevant to <strong>the</strong> assessment challenges librarians face as <strong>the</strong>y 

demonstrate <strong>the</strong> library’s impact on institutions <strong>of</strong> higher education. These questions 

are “(1) How committed are librarians to student learning? (2) What do librarians want 

students to learn? (3) How do librarians document student learning? (4) How committed 

are librarians to <strong>the</strong>ir own learning? (5) What do librarians need to learn? (6) How 

can librarians document <strong>the</strong>ir own learning?” After answering <strong>the</strong>se questions, Oakleaf 

concludes with <strong>the</strong> observation that librarians involved in information literacy instruction 

should use impact maps and assessment plans to assist <strong>the</strong>m in demonstrating 

that <strong>the</strong> student learning-outcome goals for <strong>the</strong> courses and sessions <strong>the</strong>y are teaching 

intersect with <strong>the</strong>ir university or college outcome goals, or with accreditation needs, 

goals, outcomes, and standards. She believes that assessment management s<strong>of</strong>tware 

will facilitate <strong>the</strong> accomplishment <strong>of</strong> this effort. 23 

Joseph Mat<strong>the</strong>ws examines <strong>the</strong> challenges involved in demonstrating organizational 

effectiveness and <strong>the</strong> role <strong>of</strong> performance measures as a way <strong>of</strong> demonstrating effectiveness. 

24 In <strong>the</strong> final article <strong>of</strong> <strong>the</strong> January 2011 special issue <strong>of</strong> Library Quarterly, Stephen 

Town examines <strong>the</strong> belief that libraries must prove <strong>the</strong>ir worth and points out that, if this 

belief is valid, library leaders may not be achieving this goal satisfactorily. He suggests 

that in <strong>the</strong> future pro<strong>of</strong> <strong>of</strong> a library’s worth should be measured by what higher order 

beneficial effects libraries provide and that evaluation using current performance frames 

<strong>of</strong> reference are insufficient. He would like to see a “new, higher order framework for 

evaluation and performance measurement based on a values scorecard.” 25 

Gail Staines reports in an article published in Library Management in 2009 on research 

she had undertaken to determine <strong>the</strong> level <strong>of</strong> an academic library’s strategic credibility 

by discovering whe<strong>the</strong>r <strong>the</strong> library’s strategic planning goals and annual reports were in 

alignment. She analyzed <strong>the</strong> strategic plans and annual reports <strong>of</strong> 28 randomly selected 

institutions belonging to <strong>the</strong> Association <strong>of</strong> Research Libraries. Her research found that 

<strong>the</strong> majority <strong>of</strong> <strong>the</strong> libraries in <strong>the</strong> study produced strategic plans. However, because 

most <strong>of</strong> those libraries no longer produced annual reports, Staines found that determining 

whe<strong>the</strong>r ARL libraries had strategic credibility was problematic. Interestingly, 

her study did reveal that Canadian ARL libraries included in <strong>the</strong> study tended to have 

strategic plans that were user-centered, whereas United States ARL libraries tended to 

have plans that focused on “hot topics.” 26 

Martha Kyrillidou and Colleen Cook review <strong>the</strong> development <strong>of</strong> measurement and 

evaluation in academic and research libraries in an article published in 2008. They review 

<strong>the</strong> topic from <strong>the</strong> perspectives <strong>of</strong> three individuals who significantly influenced library 

assessment activities over <strong>the</strong> years, James Gerould, F. Wilfrid Lancaster, and Duane Webster. 

27 Maria Pinto writes about library service charters in 2008. Quality issues reported in 

<strong>the</strong> pr<strong>of</strong>essional literature were used as resources in developing a checklist. The checklist 



was used to assess randomly selected service charters. The project disclosed that many 

<strong>of</strong> <strong>the</strong> academic library service charters studied needed considerable improvement. 28 

Several papers on various aspects <strong>of</strong> academic and research library assessment are 

included in <strong>the</strong> Proceedings <strong>of</strong> <strong>the</strong> 2008 Library Assessment Conference held in Seattle. 

Steve Hiller and Stephanie Wright examine how assessment information has been used 

to make changes in North American academic research libraries. 29 The information came 

from two Association <strong>of</strong> Research Libraries resources—<strong>the</strong> Making Library Assessment 

Work/Effective, Sustainable, and Practical Assessment service and SPEC Kit 303: Library Assessment. 

30 The authors review <strong>the</strong> assessment methods used by both resources to acquire 

<strong>the</strong> information and <strong>the</strong> changes or improvements made by <strong>the</strong> libraries as a result <strong>of</strong> 

<strong>the</strong> information. In ano<strong>the</strong>r paper in <strong>the</strong> Proceedings, Stephanie Brasley, Penny Beile, 

and Irv Katz discuss use <strong>of</strong> <strong>the</strong> iSkills assessment tool in several assessment projects that 

were conducted throughout <strong>the</strong> California 

State University (CSU) system and at <strong>the</strong> 

The iSkills tool is a product <strong>of</strong> <strong>the</strong> 

University <strong>of</strong> Central Florida (UCF). The 

iSkills tool is a product <strong>of</strong> <strong>the</strong> collaborative 

efforts <strong>of</strong> academic librarians across 

collaborative efforts <strong>of</strong> academic 

<strong>the</strong> United States and <strong>the</strong>refore reflects a 

librarians across <strong>the</strong> United States 

and <strong>the</strong>refore reflects a national 

perspective <strong>of</strong> information literacy 

competence. 

national perspective <strong>of</strong> information literacy 

competence. The authors address iSkills’ 

purpose and development, and explain 

how <strong>the</strong> CSU system and UCF used iSkills 

to assess student learning and instructional 

competence. Also in <strong>the</strong> Proceedings is a paper by Leslie Bussert, Sue Phelps, and Karen 

Diller that examines two assessment projects—one undertaken at <strong>the</strong> University <strong>of</strong> 

Washington at Bo<strong>the</strong>ll and Cascadia Community College and <strong>the</strong> o<strong>the</strong>r at Washington 

State University at Vancouver—involving au<strong>the</strong>ntic assessment instruments. The data 

collected by <strong>the</strong>se instruments helped identify ways to improve <strong>the</strong> quality <strong>of</strong> information 

literacy learning and instruction. The projects’ findings have also assisted in improving 

librarian, faculty, and administrator collaboration, institutional development, and 

program planning on <strong>the</strong> campuses. 31 

In an article published in 2007, Larry White and Emily Blankenship discuss <strong>the</strong> benefits 

<strong>of</strong> integrating and aligning assessment process components, and that doing so will 

improve an academic library’s effectiveness. The authors explain that <strong>the</strong> components 

include assessment, capacity, stakeholders, participation, technology, and organizational 

leadership. They maintain that “libraries that can effectively align <strong>the</strong>ir assessment 

processes to <strong>the</strong> service environment in which <strong>the</strong>y operate will be more successful in 

demonstrating and reporting assessment <strong>of</strong> <strong>the</strong>ir organizational performance.” 32 Amos 

Lakos writes about <strong>the</strong> use <strong>of</strong> data in library decision-making and specifically about 

<strong>the</strong> role <strong>of</strong> library leaders in encouraging evidence-based decisions in ano<strong>the</strong>r article 

published in 2007. He explains that developments in information technology, especially 

<strong>the</strong> growth <strong>of</strong> large networked infrastructures, digitization projects, and collaboration 

along with economic and market trends, have <strong>the</strong> potential to greatly increase <strong>the</strong> use 

<strong>of</strong> data analysis by library management. The article includes discussion <strong>of</strong> several new 

products and services designed to assist decision-makers and also interviews with over 

twenty library directors. 33 


14 


In an effort to determine how <strong>the</strong> adequacy <strong>of</strong> library budgets might be assessed, 

Frank Allen and Mark Dickie examine whe<strong>the</strong>r a relationship between academic library 

funding and certain institutional factors could indicate demand for library services. 

The factors include 

enrollment, number 

<strong>of</strong> doctoral programs, 

doctoral degrees 

awarded, and number 

<strong>of</strong> faculty. The authors 

evaluated eleven 

years <strong>of</strong> longitudinal data from 113 ARL libraries. The study’s results revealed that <strong>the</strong> 

indicators <strong>of</strong> demand for library services were in fact positively associated with funding. 

The authors maintain that <strong>the</strong> study’s methodology may be useful as a tool in determining 

an appropriate level <strong>of</strong> library funding and for assessing a library budget’s adequacy. 34 

Ronald Powell provides an overview <strong>of</strong> evaluation research in an article published 

in <strong>the</strong> summer 2006 issue <strong>of</strong> Library Trends. The article first examines <strong>the</strong> reasons for 

conducting evaluation research and <strong>the</strong>n reviews <strong>the</strong> principles and types <strong>of</strong> research. 

Several evaluation methods are examined, including “input measurement, output/ 

performance measurement, impact/outcomes assessment, service quality assessment, 

process evaluation, benchmarking, standards, quantitative methods, qualitative methods, 

cost analysis, organizational effectiveness, and program evaluation methods.” The steps 

involved in planning and conducting a study are discussed along with <strong>the</strong> measurement 

process, statistics, and data collection techniques. In addition, <strong>the</strong> article includes 

a review <strong>of</strong> data analysis and <strong>the</strong> evaluation report. 35 

Cost accounting methods can be used to make better decisions about what library 

services to <strong>of</strong>fer users according to Roswitha Poll. In an article published in 2006, she 

defines cost accounting and points out <strong>the</strong> difference between it and <strong>the</strong> traditional statistics 

<strong>of</strong> income and expenses. She maintains that “by knowing <strong>the</strong> cost <strong>of</strong> specific tasks 

or services and comparing those costs with similar services <strong>of</strong>fered by o<strong>the</strong>r institutions, 

<strong>the</strong> library can better judge what services should be <strong>of</strong>fered.” She also avers that cost 

accounting should always support a library’s mission, goals, and strategic planning. 36 

Ano<strong>the</strong>r article authored by Poll along with Philip Payne provides an overview <strong>of</strong> impact 

(a.k.a. outcomes) research and a selection <strong>of</strong> project experiences illustrates this kind <strong>of</strong> 

research. The authors state that impact research can be employed to prove that use <strong>of</strong> 

library services positively influenced users. They go on to point out that “benefits from 

library services can be assessed in terms <strong>of</strong> knowledge gained, greater information literacy, 

greater academic or pr<strong>of</strong>essional success, or increases in individual well-being,” 

and conclude that, since existing methods are inadequate for measuring influence, new 

research methods must be developed to show a library’s overall impact. 37 

Hiller, Kyrillidou, and Self grade <strong>the</strong> progress <strong>of</strong> research library assessment. Their 

The study’s results revealed that <strong>the</strong> indicators <strong>of</strong> 

demand for library services were in fact positively 

associated with funding. 


article reports <strong>the</strong> results <strong>of</strong> <strong>the</strong> first part <strong>of</strong> a two-year project sponsored by <strong>the</strong> Association 

<strong>of</strong> Research Libraries. The project provides librarians with information <strong>the</strong>y need 

to make appropriate decisions in selecting and applying measurement techniques and 

on how to use assessment data in <strong>the</strong>ir decision-making. The article supports “moving 

library assessment from a project-based approach to a more programmatic, integrated,


and sustainable operation within libraries.” 38 Xi Shi and Sarah Levy examine <strong>the</strong>oretical 

models used in library assessment activities. Their examination includes a review <strong>of</strong> <strong>the</strong> 

history <strong>of</strong> library assessment practices and <strong>the</strong> evolution <strong>of</strong> <strong>the</strong> various approaches. The 

focus is on service quality and user satisfaction. 39 

LibQUAL+ TM 

Much has been written about LibQual+ TM . The Association <strong>of</strong> Research Libraries has 

an extensive bibliography <strong>of</strong> significant papers, presentations, and articles that review 

employments or analyses <strong>of</strong> LibQual+ TM . 40 These publications are well worth reading. 

LibQUAL+ TM comprises a package <strong>of</strong> instruments by which libraries can determine 

<strong>the</strong>ir users’ opinions <strong>of</strong> <strong>the</strong>ir service quality, and also market <strong>the</strong> library. The main instrument 

is a Web-based survey. LibQUAL+ TM began as a research and development project 

undertaken in 2003 by <strong>the</strong> Association 

<strong>of</strong> Research Libraries (ARL) in collaboration 

with Texas A&M University and 

LibQUAL+ TM comprises a package <strong>of</strong> 

with financial support from <strong>the</strong> United instruments by which libraries can 

States Department <strong>of</strong> Education’s Fund 

determine <strong>the</strong>ir users’ opinions <strong>of</strong> 

for <strong>the</strong> Improvement <strong>of</strong> Postsecondary 

Education. Libraries that implement 

<strong>the</strong> survey can refer to <strong>the</strong> results when 

describing <strong>the</strong>ir contributions to <strong>the</strong>ir 

institutions. Based on Gap <strong>the</strong>ory, <strong>the</strong> 

LibQUAL+ TM survey measures library users’ perceptions <strong>of</strong> service quality and identifies 

gaps between minimum, desired, and perceived expectations <strong>of</strong> service. LibQUAL+ TM 

is modeled on SERVQUAL, a service quality assessment tool that was introduced to <strong>the</strong> 

commercial sector in 1988 by its authors A. Parasuraman, L.L. Berry, and V.A. Zeitkaml. 41 

SERVQUAL began to be employed by libraries in <strong>the</strong> 1990s. 

The authors <strong>of</strong> two articles also worth reading criticized SERVQUAL and 

LibQUAL+ TM for limitations on <strong>the</strong> validity <strong>of</strong> <strong>the</strong>ir methods. Liangzhi Yu and o<strong>the</strong>rs 

review a study that tested SERVQUAL at three Chinese university libraries. Based on 

<strong>the</strong> findings <strong>of</strong> this study, <strong>the</strong> authors questioned a number <strong>of</strong> assumptions that <strong>the</strong>y 

claim limited SERVQUAL’s ability to yield valid results. They suggest three enhancements 

to make it more useful as a library assessment tool. William Edgar <strong>of</strong>fers criticism 

<strong>of</strong> LibQUAL+ TM . He describes <strong>the</strong> LibQUAL+ TM survey instrument and its research 

approach and points out <strong>the</strong> fundamental assumption on which LibQUAL+ TM rests, 

which is that only users can judge quality. Edgar proposes what he calls a “functional/ 

technical” model for measuring academic library effectiveness that would address what 

he sees as LibQUAL+ TM ’s shortcomings. This model recognizes that effectiveness also 

depends on such things as a library’s operations, <strong>the</strong> groups it serves, <strong>the</strong> values provided 

to <strong>the</strong>m, <strong>the</strong> models for funding libraries, <strong>the</strong> obligations <strong>of</strong> academic libraries, 

and <strong>the</strong> library users’ need for pr<strong>of</strong>essional assistance. According to Edgar, “this broader 

approach provides a basis for demonstrating both immediate and cumulative academic 

library effectiveness.” 42 

<strong>the</strong>ir service quality, and also market 

<strong>the</strong> library. 


16 


Reports on use <strong>of</strong> <strong>the</strong> results <strong>of</strong> <strong>the</strong> LibQUAL+ TM survey and <strong>of</strong> follow-up assessments 

to it can be found in <strong>the</strong> literature. John Harer describes how small academic 

libraries may benefit from using LibQUAL+ TM . 43 He points out that <strong>the</strong> benefits may 

vary depending on <strong>the</strong> size <strong>of</strong> <strong>the</strong> library. Melissa Becher and Janice Flug report that, 

following implementation <strong>of</strong> <strong>the</strong> LibQUAL+ TM survey at <strong>the</strong>ir university library, followup 

assessment took place using three focus group sessions. 44 The authors explain <strong>the</strong> 

methodology, implementation, and results <strong>of</strong> <strong>the</strong> focus group study. They also describe 

how <strong>the</strong> findings <strong>of</strong> <strong>the</strong> LibQUAL+ TM survey, <strong>the</strong> focus groups, and o<strong>the</strong>r assessment 

activities were used as resources in <strong>the</strong>ir library’s planning and marketing efforts. 

Kathleen Miller’s dissertation “Service Quality in Academic Libraries: An Analysis 

<strong>of</strong> LibQUAL+ TM Scores and Institutional Characteristics” reports <strong>the</strong> findings <strong>of</strong> a 

study that sought to determine whe<strong>the</strong>r service quality scores from <strong>the</strong> LibQUAL+ TM 

instrument were related to <strong>the</strong> type <strong>of</strong> institution, size <strong>of</strong> <strong>the</strong> institution, or <strong>the</strong> level <strong>of</strong> 

<strong>the</strong> institution’s investment in its libraries. Data regarding Carnegie classification, FTE 

enrollment, and library expenditures at 159 colleges and universities participating in <strong>the</strong> 

LibQUAL+ TM survey <strong>of</strong> 2006 were examined. Several statistically significant relationships 

were found, most notably, negative correlations between <strong>the</strong> LibQUAL+ TM scores and 

total library expenditures. The study’s findings also suggested that higher expectations 

among library users in large research libraries led to slightly lower LibQUAL+ TM scores. 45 

The Digital Revolution 

The two Ithaka reports discussed in this section provide important contributions to an 

understanding <strong>of</strong> how <strong>the</strong> digital revolution is affecting academic and research libraries. 

Also, Transforming Library Service 

through Information Commons: Case 

Studies for <strong>the</strong> Digital Age is well 

worth reading for insight into 

how some libraries are using <strong>the</strong> 

new technology to transform <strong>the</strong>ir 

services. 

The change from older analog 

technology to digital technology 

that has been taking place since <strong>the</strong> early 1980s is <strong>of</strong>ten referred to as <strong>the</strong> digital revolution. 

The information age is ano<strong>the</strong>r term used for this period <strong>of</strong> change. It refers to <strong>the</strong> 

sweeping changes brought about by digital computing and communication technology 

that includes, among o<strong>the</strong>r things, <strong>the</strong> 

computer, digital cellular phone, and fax 

machine. Studies have been published 

that examine <strong>the</strong> effect <strong>of</strong> <strong>the</strong> digital revolution 

on academic and research libraries. 

Ithaka is a nonpr<strong>of</strong>it organization 

dedicated to helping <strong>the</strong> academic community 

use digital technologies to preserve 

<strong>the</strong> scholarly record and advance 

The two Ithaka reports discussed in this 

section provide important contributions 

to an understanding <strong>of</strong> how <strong>the</strong> digital 

revolution is affecting academic and 

research libraries. 

Ithaka is a nonpr<strong>of</strong>it organization 

dedicated to helping <strong>the</strong> academic 

community use digital technologies 

to preserve <strong>the</strong> scholarly record and 

advance research and teaching. 



research and teaching. Reports published by Ithaka assist in identifying critical issues 

facing academe. A recent Ithaka report titled Ithaka S+R Library Survey 2010: Insights from 

U.S. Academic Library Directors examines a survey <strong>of</strong> 267 high level library administrators 

at four-year colleges and universities to determine <strong>the</strong> strategic direction <strong>the</strong> administrators 

were planning for <strong>the</strong>ir libraries. Among several key findings was <strong>the</strong> belief on 

<strong>the</strong> part <strong>of</strong> most <strong>of</strong> <strong>the</strong>se administrators that support for research and teaching is very 

important and that <strong>the</strong>ir libraries would experience a continued increase in <strong>the</strong> amount 

spent on digital materials and a commensurate decrease for print materials. Also, <strong>the</strong>y 

expected that by 2015 <strong>the</strong>ir libraries will have completed <strong>the</strong> transition to electronic 

format for journals acquisitions and at <strong>the</strong> same time will be spending nearly half <strong>the</strong>ir 

books budget on electronic books. 46 The findings <strong>of</strong> two large-scale surveys are reported 

in Ithaka’s 2006 Studies <strong>of</strong> Key Stakeholders in <strong>the</strong> Digital Transformation in Higher Education. 

The purpose <strong>of</strong> <strong>the</strong> authors, Ross Housewright and Roger Schonfeld, in writing this report 

was to assist librarians and o<strong>the</strong>r information pr<strong>of</strong>essionals interested in scholarship in 

<strong>the</strong> digital age to understand <strong>the</strong> changing needs <strong>of</strong> faculty and to encourage thoughtful 

consideration <strong>of</strong> how to best serve <strong>the</strong>m in this changing environment. The surveys 

questioned faculty to find out how <strong>the</strong>y felt about <strong>the</strong> transition to a digital environment. 47 

A collection <strong>of</strong> case studies regarding successful programs for information commons 

development is included in Transforming Library Service through Information Commons: 

Case Studies for <strong>the</strong> Digital Age. Russell Bailey and Barbara Tierney authored <strong>the</strong> book, and 

<strong>the</strong> American Library Association published it in 2008. The authors saw <strong>the</strong> information 

commons as an important aspect <strong>of</strong> <strong>the</strong> new digital age, and wrote <strong>the</strong> book with <strong>the</strong> 

expectation that it would be used as a practical guide for encouraging and designing 

effective change in facilities, informational resources, services, and staff. They identified 

<strong>the</strong>se selected cases as creations <strong>of</strong> “intelligent library and information pr<strong>of</strong>essionals 

determined to provide access to dynamic, vital, and evolutionary teaching and learning 

laboratories in <strong>the</strong>ir particular iterations <strong>of</strong> <strong>the</strong> commons.” Chapter 5 discusses assessment 

<strong>of</strong> information commons. 48 

Richard Johnson and Judy Lu<strong>the</strong>r authored a 2007 Association <strong>of</strong> Research Libraries 

(ARL) report that examines <strong>the</strong> issues associated with <strong>the</strong> move toward <strong>the</strong> digitization 

<strong>of</strong> <strong>the</strong> content <strong>of</strong> journals. The report is based on information collected during interviews. 

Among those interviewed were collection development librarians at a dozen 

ARL libraries. Publishing staff <strong>of</strong> societies and university presses, publishing platform 

hosts, and publishing production consultants were also interviewed. The study sought 

to discover how far <strong>the</strong> process <strong>of</strong> changing to digital journal publishing had advanced 

and to examine perceptions <strong>of</strong> replacing print with entirely electronic journals. 49 

Lloyd Davidson provides a nice overview <strong>of</strong> <strong>the</strong> transformation from print to digital 

media. He discusses <strong>the</strong> growth <strong>of</strong> <strong>the</strong> Internet, <strong>the</strong> explosion in <strong>the</strong> amount <strong>of</strong> information 

now available to everyone, and <strong>the</strong> fundamental changes in institutions that exist 

for <strong>the</strong> creation, management, and distribution <strong>of</strong> information and knowledge. He points 

out that change brought about by digitization is affecting all components <strong>of</strong> <strong>the</strong> scientific 

enterprise, from personal correspondence and laboratory methods to peer reviewing and 

<strong>the</strong> quality assessment <strong>of</strong> scientific research. The consequence <strong>of</strong> this revolution threatens 

long established institutions such as copyright, commercial publishers, scientific societies, 

and academic libraries, according to Davidson. 50 


18 


Needs Assessment and User Surveys 

Two sources, discussed below, provide significant contributions in <strong>the</strong> areas <strong>of</strong> needs 

assessment and user surveys. Char Booth’s report <strong>of</strong> a large-scale environmental scan 

done at Ohio University that tested librarian assumptions <strong>of</strong> what library users wanted 

to see in new public service technologies by surveying student perceptions and attitudes 

had important implications for technology development and evaluation for libraries in 

general. Studying Students: <strong>the</strong> Undergraduate Research Project at <strong>the</strong> University <strong>of</strong> Rochester, 

by Nancy Foster and Susan Gibbons, introduced to <strong>the</strong> library pr<strong>of</strong>ession user studies 

that involved anthropological and ethnographic methods. Since <strong>the</strong>ir report was published 

in 2007, o<strong>the</strong>r academic and research libraries have followed suit and conducted 

similar studies. 

Needs assessment entails a systematic exploration <strong>of</strong> current and desired conditions 

in an organization with <strong>the</strong> goal <strong>of</strong> improvement. A user survey, on <strong>the</strong> o<strong>the</strong>r hand, 

is designed to collect information from a targeted population <strong>of</strong> users on something 

such as services or resources. Shun 

Wong and T. Webb describe how 

Hong Kong Baptist University 

Library conducted a user survey 

that established a relationship between 

students’ library material use 

and <strong>the</strong>ir cumulative grade point 

average. 51 The study population 

included 2007 to 2009 graduates. 

Needs assessment entails a systematic 

exploration <strong>of</strong> current and desired 

conditions in an organization with <strong>the</strong> 

goal <strong>of</strong> improvement. A user survey, on 

<strong>the</strong> o<strong>the</strong>r hand, is designed to collect 

information from a targeted population 

<strong>of</strong> users on something such as services 

or resources. 

The data collected included grade 

point averages and library book 

and audiovisual loans. The authors 

maintain that analysis <strong>of</strong> <strong>the</strong> data 

revealed <strong>the</strong> Library’s impact on 

student learning outcomes. The subjects were grouped into disciplines according to <strong>the</strong>ir 

declared majors at graduation, and, <strong>of</strong> <strong>the</strong> 48 groups, 31 (65 percent) showed a positive 

correlation between grade point average and <strong>the</strong> number <strong>of</strong> library loans. 

Char Booth, in collaboration with <strong>the</strong> Ohio University Libraries Technology Team, 

led a large-scale environmental scanning initiative in <strong>the</strong> winter and spring <strong>of</strong> 2008 that 

investigated <strong>the</strong> factors motivating student interest in emerging library technologies. 

In particular, <strong>the</strong> initiative assessed student library, information, and communication 

technology use. It consisted <strong>of</strong> two Web-based surveys, both <strong>of</strong> which generated strong 

response rates. By researching student library perceptions in conjunction with <strong>the</strong>ir 

attitudes toward information, communication, and academic tools, Booth and her colleagues 

hoped to test <strong>the</strong> assumptions that influenced <strong>the</strong> Ohio University Libraries’ 

development <strong>of</strong> emerging public service technologies. The initiative came about as <strong>the</strong> 

result <strong>of</strong> OU librarians realizing that <strong>the</strong>y needed to understand how students actually 

interacted with libraries and technology, instead <strong>of</strong> how librarians assumed that <strong>the</strong>y did. 

Booth explains in her report <strong>of</strong> <strong>the</strong> project how libraries can use collected data to both 

evaluate and prioritize a range <strong>of</strong> initiatives. 52 She points out that, though <strong>the</strong> surveys’ 



findings are specific to <strong>the</strong> population and culture <strong>of</strong> Ohio University Libraries, <strong>the</strong>y 

<strong>of</strong>fer implications for technology development and evaluation for o<strong>the</strong>r libraries as well. 

She also provides insight into <strong>the</strong> various types <strong>of</strong> research, pointing out <strong>the</strong>ir strengths 

and applications in a library context, provides a sample survey instrument template 

modeled on <strong>the</strong> surveys used in <strong>the</strong> OU Libraries scanning initiative, and recommends 

additional data sources that can inform local research initiatives. 

Joanna Bryant, Graham Mat<strong>the</strong>ws, and Graham Walton describe a study done at 

Loughborough University Library (in <strong>the</strong> United Kingdom) that evaluated <strong>the</strong> use <strong>of</strong> 

an open learning/social space in <strong>the</strong> Library. 53 The 

study employed a descriptive ethnographic method 

to collect data. The data was examined as it related 

The study employed a 

to <strong>the</strong> <strong>the</strong>mes <strong>of</strong> “collaborative study, individual descriptive ethnographic 

study, social space, intrusions and interruptions, 

method to collect data. 

use <strong>of</strong> technology, diversity, library staff/library 

materials, and spatial organization.” 

In an article published in 2008, Kathleen Murray and Inga Hsieh describe a needs 

assessment study undertaken as part <strong>of</strong> <strong>the</strong> Library <strong>of</strong> Congress-funded Web-at-Risk 

project. The study was a three-year collaborative research effort on <strong>the</strong> part <strong>of</strong> <strong>the</strong> California 

Digital Library, <strong>the</strong> University <strong>of</strong> North Texas, and New York University to identify 

<strong>the</strong> needs and concerns <strong>of</strong> librarians, curators, end users, and content providers about 

Web archives. Twenty-two curators, 43 librarians and archivists, seven researchers, and 

seven content providers participated in activities that included an online survey, focus 

groups, and interviews. One important outcome <strong>of</strong> <strong>the</strong> study was that <strong>the</strong> participating 

curators developed a Web archiving service for materials taken mostly from federal and 

state government agency websites. 54 

A major needs assessment trend over <strong>the</strong> past five years involved <strong>the</strong> use <strong>of</strong> anthropological 

methods. These methods were introduced into <strong>the</strong> mainstream <strong>of</strong> library 

research in 2007 by <strong>the</strong> authors Nancy Foster and Susan Gibbons. In Studying Students: 

The Undergraduate Research Project at <strong>the</strong> University <strong>of</strong> Rochester, Foster and Gibbons 

reported on a research project undertaken by <strong>the</strong> University <strong>of</strong> Rochester’s Library 

that determined what undergraduates “really” do when <strong>the</strong>y research and write <strong>the</strong>ir 

papers. 55 The project’s researchers felt <strong>the</strong>y needed to know more about present-day 

student research habits, <strong>the</strong>ir library-related assignments, and library-related needs in 

order for librarians to meet <strong>the</strong> University’s education mission. They <strong>the</strong>refore decided 

to develop and employ a study involving anthropological and ethnographic methods 

that would help <strong>the</strong>m find out what <strong>the</strong> undergraduates were doing to complete <strong>the</strong>ir 

research paper assignments. 

Penn State University’s Visual Image User Study is described in an article by Henry 

Pisciotta and o<strong>the</strong>rs that was published in portal: Libraries and <strong>the</strong> Academy in 2005. The 

study was a needs assessment <strong>of</strong> users <strong>of</strong> pictures at Penn State and <strong>of</strong> <strong>the</strong>ir perceptions. 

The findings included identification <strong>of</strong> <strong>the</strong> potential market for digital images and <strong>the</strong> 

factors determining whe<strong>the</strong>r an image management system will be used. Also identified 

were key user requirements for images in teaching, independent learning, and collection 

management. 56 


20 


Information Literacy 

Important sources for information literacy that are discussed in this section include <strong>the</strong> 

Middle States Commission on Higher Education’s Student Learning Assessment: Options 

and Resources and Information Literacy Assessment: Standards-based Tools and Assignments 

by Teresa Neely. Also, Megan Oakleaf’s article “The Information Literacy Instruction 

Assessment Cycle: A Guide for Increasing Student Learning and Improving Librarian 

Instructional Skills” and John Budd’s article in which he presents an alternative approach 

to that <strong>of</strong>fered by ACRL’s Information Literacy Competency Standards for Higher Education 

are important contributions to <strong>the</strong> literature, and are well worth reading. 

Jeremy Shapiro and Shelley Hughes define information literacy as “a new liberal 

art that extends from knowing how to use computers and access information to critical 

reflection on <strong>the</strong> nature <strong>of</strong> information itself, its social, cultural, and philosophical context 

and impact.” 57 The Final Report <strong>of</strong> <strong>the</strong> American Library Association’s Presidential 

Committee on Information Literacy, a document that inaugurated information literacy 

programs in academic and research libraries, states that “to be information literate, a 

person must be able to recognize when information is needed and has <strong>the</strong> ability to locate, 

evaluate, and use effectively <strong>the</strong> needed information.” 58 In recent years, librarians 

in academic and research libraries have embraced information literacy instruction as 

one <strong>of</strong> <strong>the</strong> most important services <strong>the</strong>ir libraries provide, and much has been published 

on assessment in this area. 

The sources examined in this section, like those in <strong>the</strong> o<strong>the</strong>r sections, are organized 

by format and within format by date <strong>of</strong> publication with <strong>the</strong> most recently published 

sources treated first. These sources include handbooks, manuals, and guides, ei<strong>the</strong>r for 

assessing information literacy programs, courses, and one-shot sessions or assisting in 

<strong>the</strong> implementation <strong>of</strong> accreditation 

standards. Also included are brief 

reviews <strong>of</strong> articles on assessment 

projects that used pre- and postsurveys, 

portfolio assignments, 

The sources examined in this section, 

like those in <strong>the</strong> o<strong>the</strong>r sections, are 

organized by format and within format 

by date <strong>of</strong> publication with <strong>the</strong> most 

recently published sources treated first. 

grading rubrics, a qualitative instrument 

for assessing critical reflection, 

a self-assessment instrument, and 

one-on-one and focus group interviews. 

Coverage also includes articles on <strong>the</strong> assessment cycle, speculations on <strong>the</strong> future 

<strong>of</strong> information literacy, concerns about <strong>the</strong> inadequacy <strong>of</strong> <strong>the</strong> ACRL Information Literacy 

Competency Standards, an examination <strong>of</strong> <strong>the</strong> degree to which <strong>the</strong> United States regional 

accreditation organizations have incorporated information literacy into <strong>the</strong>ir standards, 

continuous assessment <strong>of</strong> a program at multiple times and using several instruments, 

integrating information literacy into a general education program, information literacy 

test development, information literacy needs <strong>of</strong> international students, assessing online 

syllabi for library skills courses using <strong>the</strong> ACRL Standards as a framework for categorizing 

<strong>the</strong>m, and issues relating to government policies and practices that affect information 

literacy. The section concludes with a brief description <strong>of</strong> a dissertation that assessed 

<strong>the</strong> readiness <strong>of</strong> graduate students in teacher education to integrate information literacy 

into <strong>the</strong>ir classroom teaching. 



The Middle States Commission on Higher Education’s publication Student Learning 

Assessment: Options and Resources exemplifies <strong>the</strong> efforts <strong>of</strong> <strong>the</strong> regional accreditation 

organizations to encourage <strong>the</strong> growth <strong>of</strong> student learning-outcomes assessment in 

higher education, including in information literacy. The publication is a handbook meant 

to serve as a resource for universities and colleges needing assistance in implementing 

<strong>the</strong> Commission’s standards for accreditation in a campus environment not conducive 

to assessment and continuous improvement. The handbook clarifies <strong>the</strong> principles and 

methods for setting goals for student learning in <strong>the</strong> context <strong>of</strong> an institution’s mission. 

It provides assistance in using <strong>the</strong> methods chosen by <strong>the</strong> institution for evaluating <strong>the</strong> 

achievement <strong>of</strong> <strong>the</strong> goals and in using <strong>the</strong> information ga<strong>the</strong>red as a resource in <strong>the</strong> effort 

toward continuous improvement in student learning. The Middle States Commission 

on Higher Education recognizes more than most regional accreditation organizations 

that information literacy has a critical role to play in <strong>the</strong> process <strong>of</strong> enhancing student 

learning, and this position is reflected in <strong>the</strong> Commission’s standards. 59 

Handbooks, manuals, and guides for information literacy assessment abound. Assessment 

<strong>of</strong> information literacy instruction for particular disciplines has been written 

about in at least one monograph in recent years. Dorothy Warner authored a book that 

includes what she refers to as “proposals,” each pertinent to a particular discipline and 

providing a framework for teaching and assessing student information literacy skills 

within that discipline. Each proposal includes a cognitive sequence related to <strong>the</strong> courses 

<strong>of</strong> a disciplinary program. The sequence begins with introductory skills and advances 

to more sophisticated cognitive skills. The learning objectives <strong>of</strong> <strong>the</strong> information literacy 

segments <strong>of</strong> <strong>the</strong> courses are adapted from <strong>the</strong> ACRL Information Literacy Competency 

Standards for Higher Education, and <strong>the</strong>y serve as <strong>the</strong> basis for assessing <strong>the</strong> student 

learning outcomes. 60 

In <strong>the</strong> introductory chapter <strong>of</strong> a monograph authored in 2007, Carolyn Radcliff and 

o<strong>the</strong>rs maintain that <strong>the</strong>ir guide gives advice “to academic librarians and o<strong>the</strong>rs who 

want to ascertain <strong>the</strong> information literacy levels <strong>of</strong> <strong>the</strong>ir students.” The guide is designed 

for librarians and administrators who want to undertake assessment ei<strong>the</strong>r for an entire 

institution, a program, or a one-shot instructional session. A significant portion <strong>of</strong> <strong>the</strong> 

guide covers nine assessment tools. Each tool segment includes explanations, definitions, 

required resources, instructions, and examples. 61 

Carol Y. Neely, author <strong>of</strong> Information Literacy Assessment: Standards-based Tools and 

Assignments, envisions her book as a guide for building a culture <strong>of</strong> information literacy 

assessment at all levels <strong>of</strong> teaching, including one-shot sessions, credit courses, and 

programs. The book emphasizes <strong>the</strong> importance <strong>of</strong> using uniform criteria for assessing 

<strong>the</strong> information literacy skills <strong>of</strong> college and university students. Also, Neely maintains 

that <strong>the</strong> book uses <strong>the</strong> ACRL Information Literacy Competency Standards for Higher 

Education as “a framework for developing and implementing an information literacy 

research agenda.” 62 Ano<strong>the</strong>r resource belonging to <strong>the</strong> category <strong>of</strong> handbooks, manuals, 

and guides is Creating a Comprehensive Information Literacy Plan: A How-To-Do-It-Manual 

and CD-ROM for Librarians, published in 2005. Included in its overall discussion on 

creating a plan are segments dealing with assessment. 63 

Several articles have reported on projects that assessed information literacy instruction 

at <strong>the</strong> program, course, or session level since 2005. Yu-Hui Chen and Mary Van Ullen 


22 


analyze <strong>the</strong> results <strong>of</strong> assessments <strong>of</strong> workshops on <strong>the</strong> research process and plagiarism 

given to international students at <strong>the</strong> University <strong>of</strong> Albany. The assessments included 

pre- and post-tests and interviews <strong>of</strong> <strong>the</strong> students several months after <strong>the</strong> workshops 

were given. The authors assert <strong>the</strong> assessments revealed that <strong>the</strong> students significantly 

improved <strong>the</strong>ir understanding <strong>of</strong> <strong>the</strong> research process and plagiarism by attending <strong>the</strong> 

workshops, and continued to apply new skills months later. 64 

Jon Hufford writes about <strong>the</strong> experience gained by librarians at Texas Tech University 

Libraries while developing and implementing pre- and post-assessment surveys 

administered in eleven sections <strong>of</strong> a library research course taught in <strong>the</strong> fall <strong>of</strong> 2008. 65 

In addition, he and Arlene Paschel authored an article reporting <strong>the</strong> results <strong>of</strong> pre- and 

post-surveys administered in <strong>the</strong> distance learning section <strong>of</strong> <strong>the</strong> same course taught in 

<strong>the</strong> fall <strong>of</strong> 2009. The assessments’ findings were used to improve both <strong>the</strong> content and 

<strong>the</strong> teaching <strong>of</strong> <strong>the</strong> course. 66 Brian Winterman writes about how he, biology researchers, 

and pr<strong>of</strong>essors collaborated in developing a course to teach biology undergraduates 

information literacy skills applicable to <strong>the</strong> research process. Pre- and post-assessment 

surveys were administered to seventeen students (fourteen seniors and three juniors) 

to determine <strong>the</strong> impact <strong>of</strong> <strong>the</strong> course during its pilot phase. A majority <strong>of</strong> <strong>the</strong> students 

showed improvement in information literacy skills. 67 

In an article published in 2009, Megan Oakleaf describes <strong>the</strong> seven stages <strong>of</strong> <strong>the</strong> Information 

Literacy Instruction Assessment Cycle (ILIAC) and uses an assignment as an 

example to show how ILIAC provides a process for improving librarian teaching skills 

and student information literacy skills. The seven stages are: reviewing learning goals, 

identifying learning outcomes, creating learning activities, enacting learning activities, 

ga<strong>the</strong>ring data to check learning, interpreting data, and enacting decisions. 68 

Laura Saunders has written about <strong>the</strong> present and future state <strong>of</strong> information literacy. 

She maintains that technological advances and changes in <strong>the</strong> information needs 

and expectations <strong>of</strong> library users are impacting information literacy programs. The 

article discusses <strong>the</strong> results <strong>of</strong> a survey <strong>of</strong> thirteen experts who were asked to respond 

to proposed futures for information literacy over <strong>the</strong> next decade. Though generally 

optimistic about its future, <strong>the</strong> experts saw obstacles that academic librarians involved 

in instruction will face. One concern expressed was that, though librarians will have 

greater opportunities to partner more fully with faculty in instructional and assignment 

design, <strong>the</strong>y may not be adequately prepared for this role. 69 

Concerns that <strong>the</strong> Association <strong>of</strong> College and Research Libraries Information 

Literacy Competency Standards for Higher Education, along with <strong>the</strong>ir performance 

indicators, lacked some essential 

factors that should have been 

Librarians assisted teaching faculty in 

developing a method for assessing student 

learning in its general education program. 

included led John Budd to <strong>of</strong>fer 

an alternative approach. In his 

opinion, <strong>the</strong> present Standards 

are based on a syntactic model 

for working with information. 

He prefers standards based on a semantic model. Budd examines his proposed model 

in <strong>the</strong> context <strong>of</strong> a new course taught at <strong>the</strong> University <strong>of</strong> Missouri at Columbia in <strong>the</strong> 

fall semester <strong>of</strong> 2006. 70 



Karen Diller and Sue Phelps investigate <strong>the</strong> use <strong>of</strong> portfolios and rubrics in <strong>the</strong> 

beginning phase <strong>of</strong> an outcomes assessment program undertaken at Washington State 

University at Vancouver. Librarians assisted teaching faculty in developing a method 

for assessing student learning in its general education program. The program is based 

on student learning goals, including information literacy goals. The authors explain that 

online s<strong>of</strong>tware was used to evaluate <strong>the</strong> portfolios’ content. Washington State University 

at Vancouver’s information literacy program is based on ACRL best practices and 

au<strong>the</strong>ntic assessment. 71 

Kate Zoellner, Sue Samson, and Samantha Hines describe a project undertaken to 

assess <strong>the</strong> teaching and content <strong>of</strong> a module embedded in an undergraduate course, Introduction 

to Public Speaking, which is a general education requirement at <strong>the</strong> University 

<strong>of</strong> Montana. Pre- and post-surveys were used to assess student research confidence, perceptions 

<strong>of</strong> information tools, Web evaluation abilities, and assistance-seeking attitudes. 

The data from 426 student responses was analyzed to determine internal benchmarks 

and improve <strong>the</strong> University’s curricula. 72 In <strong>the</strong> summer <strong>of</strong> 2006 Kristin H<strong>of</strong>fmann and 

o<strong>the</strong>rs conducted a study at <strong>the</strong> University <strong>of</strong> Western Ontario that focused on “graduate 

student perceptions <strong>of</strong> <strong>the</strong>ir library research needs, <strong>the</strong>ir preferences for learning about 

library research, and <strong>the</strong> appropriateness <strong>of</strong> a common instruction program for students” 

majoring in engineering, <strong>the</strong> health sciences, medicine, dentistry, and science. The librarians 

used <strong>the</strong> findings <strong>of</strong> <strong>the</strong>ir study to assist <strong>the</strong>m in developing non-course-based, 

non-mandatory library instruction programs to meet <strong>the</strong> needs <strong>of</strong> graduate students. 73 

Donald Gilstrap and Jason Dupree report on <strong>the</strong> use <strong>of</strong> Brookfield’s Critical Incident 

Questionnaire at <strong>the</strong> Southwest Oklahoma State University Libraries as a qualitative instrument 

for assessing information literacy skills. A sample <strong>of</strong> 348 English Composition II 

students was studied over a period <strong>of</strong> two semesters during a four-session instructional 

program. The results <strong>of</strong> <strong>the</strong>ir study showed that <strong>the</strong> questionnaire was an effective 

instrument for assessing critical reflection. 74 Lana Ivanitskaya and o<strong>the</strong>rs review <strong>the</strong> 

results <strong>of</strong> tests that assessed <strong>the</strong> information literacy skills <strong>of</strong> <strong>of</strong>f-campus students. 75 The 

researchers used a “Research Readiness Self-Assessment” survey as a pre- and post-test 

in an <strong>of</strong>f-campus Master’s degree class at Central Michigan University. 

An examination <strong>of</strong> <strong>the</strong> degree to which <strong>the</strong> regional accreditation organizations 

incorporated and treated information literacy in <strong>the</strong>ir standards was undertaken by 

Laura Saunders. She conducted a content analysis <strong>of</strong> <strong>the</strong> organizations’ Web-based 

resources and maintains that <strong>the</strong>re is much support for information literacy and encouragement 

for partnerships between librarians and faculty in instruction and assessment. 

Saunders explains how <strong>the</strong> library can support institutional missions and accreditation 

standards. 76 An article on continuous assessment, authored by Randall Schroeder and 

Kimberly Mashek, discusses how information literacy instruction, required in five core 

courses in Wartburg College’s general education program, is assessed at multiple times 

and in multiple ways using both direct and indirect assessment. This rigorous information 

literacy assessment has persuaded “reluctant faculty and administrators to buy 

into <strong>the</strong> program.” 77 

Lynn Cameron, Steven Wise, and Susan Lottridge write about <strong>the</strong> Information Literacy 

Test (ILT) developed at James Madison University Library. 78 The test was developed 

to measure student achievement using <strong>the</strong> ACRL Information Literacy Competency 


24 


Standards for Higher Education as <strong>the</strong> basis for rating. The authors foresaw <strong>the</strong> possibility 

that <strong>the</strong> test would become more generally attractive for use at o<strong>the</strong>r institutions. The 

test makes frequent use <strong>of</strong> graphics, documents, and webpage images. Davida Scharf 

and o<strong>the</strong>rs investigate assessment <strong>of</strong> student information literacy skills using writing 

portfolios. In this study, graduating seniors taking a capstone seminar in <strong>the</strong> Humanities 

at <strong>the</strong> New Jersey Institute <strong>of</strong> Technology were required to create a writing portfolio. 

These portfolios included term papers that were assessed for this study. 79 

Because <strong>of</strong> <strong>the</strong> inclusion <strong>of</strong> information literacy in <strong>the</strong> efforts <strong>of</strong> many colleges and 

universities to assess <strong>the</strong>ir programs and courses, libraries have been able to play a 

greater role in <strong>the</strong> formal planning <strong>of</strong> general education programs and courses. Articles 

that address <strong>the</strong> efforts to integrate 

information literacy 

into general education <strong>of</strong>ten 

point out <strong>the</strong> importance <strong>of</strong> 

forming strategic alliances 

to establish information 

Because <strong>of</strong> <strong>the</strong> inclusion <strong>of</strong> information 

literacy in <strong>the</strong> efforts <strong>of</strong> many colleges and 

universities to assess <strong>the</strong>ir programs and 

courses, libraries have been able to play a 

greater role in <strong>the</strong> formal planning <strong>of</strong> general 

education programs and courses. 

literacy as a foundation for 

student learning. Thomas 

Mackey and Trudi Jacobson 

discuss <strong>the</strong> need for a strategy 

to accomplish this. 80 In 

ano<strong>the</strong>r article published in 2007, Nancy O’Hanlon explains how librarians can play a 

role in <strong>the</strong>ir institution’s student learning outcomes assessment process and reports on 

a study that determined <strong>the</strong> impact <strong>of</strong> <strong>the</strong> Ohio State University Libraries’ information 

literacy program on a number <strong>of</strong> required general education courses. 81 

Librarians pilot-tested three methods designed to assess <strong>the</strong> library instruction 

program at Cornell University’s Albert R. Mann Library in 2007. 82 The assessment instruments 

used collected data on patron attitudes about <strong>the</strong> program, learning outcomes, 

and gaps in instructor and patron attitudes toward learning outcomes for <strong>the</strong> program’s 

workshops. The authors maintain that combinations such as <strong>the</strong>se three different kinds 

<strong>of</strong> assessment methods address <strong>the</strong> shortcomings <strong>of</strong> a single method used by itself and 

provide a better measure <strong>of</strong> a library instruction program in its entirety. 

Self-assessments <strong>of</strong> information literacy skills have also been conducted. Gillian 

Gremmels and Karen Lehmann discuss a two-year study that compared student 

“self-reports” <strong>of</strong> what <strong>the</strong>y had learned during interviews at <strong>the</strong> reference desk with 

<strong>the</strong> librarian statements <strong>of</strong> what was taught and tested whe<strong>the</strong>r students perceived a 

relationship between what <strong>the</strong>y had learned during <strong>the</strong> interview and <strong>the</strong> information 

literacy content taught in a previous class. 83 Similarly, Ka<strong>the</strong>rine Schilling and Rachel 

Applegate analyze data from surveys, written tests, and a practical information literacy 

exercise to determine <strong>the</strong> extent to which students’ perception about <strong>the</strong>ir information 

literacy skills matched <strong>the</strong>ir demonstrated skills. They also examined students’ attitudes 

about <strong>the</strong> library and librarians to determine whe<strong>the</strong>r a relationship existed between 

attitudes and grades. The analysis revealed discrepancies between students’ opinions, 

self-assessments, and graded tests. 84 



Shikha Sharma reviews a study done at <strong>the</strong> University <strong>of</strong> Connecticut’s Homer Babbidge 

Library that examined <strong>the</strong> use <strong>of</strong> Web-based research portfolios in an information 

literacy course. The author suggests that <strong>the</strong> Web-based portfolio format is a very effective 

instrument for teaching research skills and for facilitating au<strong>the</strong>ntic assessment. 85 

Pamela Jackson reports on pre- and post-test assessment results that revealed students’ 

understanding <strong>of</strong> plagiarism. The students studied were Computer Science majors at San 

Jose State University. Data from 2,829 student scores was analyzed. One <strong>of</strong> <strong>the</strong> study’s 

findings was that students generally have trouble understanding paraphrasing. 86 

In “The One-Minute Paper and <strong>the</strong> One-Hour Class” Elizabeth Choinski and Michelle 

Emanuel investigate <strong>the</strong> use <strong>of</strong> an assessment technique by which students write brief 

answers to specific questions. This technique provides instant feedback from students 

on <strong>the</strong> lesson <strong>of</strong> <strong>the</strong> day. 87 In addition, Lorrie Knight discusses <strong>the</strong> use <strong>of</strong> a scoring rubric 

based on course learning objectives and <strong>the</strong> ACRL Information Literacy Competency 

Standards for Higher Education. 88 The rubric was used to score students’ course bibliographies. 

The Educational Testing Service created a revised version <strong>of</strong> its Information 

and Communication Technology Literacy Assessment (now called iSkills) in 2006. The 

assessment measures <strong>the</strong> “abilities to define, access, manage, integrate, evaluate, create, 

and communicate information.” 89 These are <strong>the</strong> skills addressed in <strong>the</strong> ACRL Information 

Literacy Competency Standards for Higher Education. 

Paul Hrycaj discusses an analysis <strong>of</strong> 100 online syllabi for introductory library skills 

courses at various institutions <strong>of</strong> higher education. They were analyzed for subject 

content, assessment techniques, and teaching methods. The ACRL Information Literacy 

Competency Standards for Higher Education were used as a framework for categorizing 

and placing in context <strong>the</strong> syllabi’s subject content. 90 Marlene Asselin, Margaret Early, 

and Margot Filipenko discuss issues relating to assessment policies and practices for <strong>the</strong> 

Canadian literacy curricula that <strong>the</strong> Canadian Ministries <strong>of</strong> Education had previously 

expanded to include new developments in information and communication technologies. 

Their examination <strong>of</strong> <strong>the</strong>se developments and prospects for <strong>the</strong> future was based 

on interviews <strong>of</strong> Ministry personnel. The authors argue for a more balanced approach 

to assessment and accountability than what <strong>the</strong> Ministry personnel had crafted. 91 

In an article published in 2005, Barry Cull discusses a study <strong>of</strong> teaching practices and 

instructional experiences among academic librarians in Canada’s Maritime Provinces. 

Instruction librarians at six universities were interviewed. The study’s results indicated 

that generally <strong>the</strong> librarians were self-reflective, student-focused, and competent teachers. 

They all believed that information literacy played a core role in academic libraries. 

Cull’s article also records <strong>the</strong> challenges and future predictions for information literacy 

as seen by <strong>the</strong> study’s participants. 92 

The impact <strong>of</strong> trends brought about by <strong>the</strong> South African government’s policies 

in higher education on academic libraries’ efforts to expand <strong>the</strong>ir information literacy 

programs is analyzed by Karin De Jager and Mary Nassimbeni. The authors review <strong>the</strong> 

progress made by librarians in creating partnerships with faculty members to deliver 

quality education and make <strong>the</strong> point that government policy initiatives in higher education 

can go a long way toward accelerating <strong>the</strong> growth and acceptance <strong>of</strong> information 

literacy education. 93 Margy MacMillan reports on a self-assessment tool called I-SKILLS 

Resume that asks students to develop individual resumes emphasizing <strong>the</strong>ir informa- 


26 


tion skills. 94 Lori Buchanan and o<strong>the</strong>rs relate how <strong>the</strong> integration <strong>of</strong> information literacy 

competencies into a first-year experience course taught at Austin Peay State University 

in Tennessee involved developing curricula, training instructors, and creating tools 

for learning-outcomes assessment. The authors emphasize planning. 95 In 2007, Tyrone 

Cannon completed a dissertation titled “Closing <strong>the</strong> Digital Divide: An Assessment <strong>of</strong> 

Urban Graduate Teacher Education Students’ Knowledge <strong>of</strong> Information Literacy and 

Their Readiness to Integrate Information Literacy into Their Teaching.” 96 

Collection Assessments 

The article authored by Robert Alan and o<strong>the</strong>rs examined in this section represents a 

significant contribution to <strong>the</strong> literature on collection development. Librarians wanting 

to learn more about collection assessment from <strong>the</strong> perspective <strong>of</strong> cost effectiveness 

should consider reading it. 

Collection development is defined as <strong>the</strong> planned purchase <strong>of</strong> materials in various 

formats to meet <strong>the</strong> instructional and research needs <strong>of</strong> a university. Assessment <strong>of</strong> a collection 

can be accomplished in different ways. Virginia Williams discusses <strong>the</strong> needs <strong>of</strong> 

education students and <strong>the</strong> characteristics <strong>of</strong> juvenile literature that academic librarians 

must be aware <strong>of</strong> when developing juvenile literature collections. 97 She examines standard 

tools for selecting children’s and young adult materials, analyzes <strong>the</strong>ir usefulness 

for building collections that support teacher education programs, and provides advice 

for evaluating juvenile collections. 

Robert Alan and o<strong>the</strong>rs describe assessments <strong>of</strong> <strong>the</strong> monograph approval plan pr<strong>of</strong>iles 

<strong>of</strong> <strong>the</strong> University <strong>of</strong> Illinois at Urbana-Champaign and Pennsylvania State University 

undertaken to answer questions about use, cost effectiveness, and coverage. 98 Data was 

collected from vendors and local online systems on books received, item circulation, 

and overlap <strong>of</strong> <strong>the</strong> two plans. The study period at both institutions was July 1, 2004, 

to June 30, 2005, for <strong>the</strong> approval plan purchasing data and July 1, 2004, to March 31, 

2007, for <strong>the</strong> circulation data. The study measured <strong>the</strong> cost effectiveness <strong>of</strong> each plan 

by subject and publisher, analyzed similarities and differences in <strong>the</strong> use <strong>of</strong> new titles, 

and examined <strong>the</strong> overlap between <strong>the</strong> two plans. The study’s primary research question 

was “Can a cost/use ratio be derived that indicates <strong>the</strong> point at which an approval 

plan pr<strong>of</strong>ile is effective or ineffective?” Mat<strong>the</strong>w Ciszek and Courtney Young examine 

<strong>the</strong> methods that can be used to assess an academic library’s collections for diversity. 99 

The methods include use <strong>of</strong> <strong>the</strong> WorldCat Collection Analysis tool, use statistics, comparing 

a collection to standard bibliographies, and <strong>the</strong> use <strong>of</strong> focus groups, interviews, 

and surveys. They point out that having a clear definition <strong>of</strong> diversity in place before 

<strong>the</strong> assessment is important. 

Public Services 


Significant sources discussed in this section include <strong>the</strong> article written by Bella Gerlich 

and Lynn Berard about a project that used <strong>the</strong> Reference Effort Assessment Data Scale to 

evaluate data collected by reference staff and <strong>the</strong> article by Corey Johnson and Elizabeth 

Lindsay that explores work-related priorities among public services librarians. Also,


Jan Brophy and David Bawden’s article that compares Google with library databases 

is interesting. 

This section examines sources on <strong>the</strong> assessment <strong>of</strong> <strong>the</strong> public services <strong>of</strong>fered by 

academic and research libraries o<strong>the</strong>r than information literacy instruction. Though 

instruction is an important service libraries 

provide <strong>the</strong>ir public, <strong>the</strong> author This section examines sources on 

decided to create a separate section for 

information literacy because <strong>of</strong> <strong>the</strong> very 

large number <strong>of</strong> publications on that <strong>of</strong>fered by academic and research 

topic. The areas <strong>of</strong> public service treated libraries o<strong>the</strong>r than information 

here include reference, including <strong>the</strong> 

use <strong>of</strong> databases, search engines, and 

literacy instruction. 

o<strong>the</strong>r kinds <strong>of</strong> electronic services to find 

information; interlibrary loan/document delivery; and loaning technology to users. One 

<strong>of</strong> <strong>the</strong> sources examined surveyed ARL public service librarians generally. 

Jim Hahn and o<strong>the</strong>rs provide an overview <strong>of</strong> <strong>the</strong> planning and implementation 

process for a technology loaning program employed at <strong>the</strong> Library <strong>of</strong> <strong>the</strong> University <strong>of</strong> 

Illinois at Urbana-Champaign. 100 The program’s purpose is to meet student needs for 

multiple technologies to support assignments and o<strong>the</strong>r projects on campus. The authors 

discuss <strong>the</strong> program’s need for policies and procedures related to acquisitions, budget 

allocation, processing, cataloging, check-out, replacement, and equipment security, as 

well as marketing <strong>the</strong> new service. The program’s comprehensive planning process 

included <strong>the</strong> use <strong>of</strong> surveys, focus groups, informal interviews, and campus conversations 

with stakeholders to ga<strong>the</strong>r information on student needs and best practices for <strong>the</strong> 

program. Analysis <strong>of</strong> <strong>the</strong> data collected led to several changes in existing services and <strong>the</strong> 

implementation <strong>of</strong> some new services, including <strong>the</strong> technology loaning program. The 

planned continuous evaluation <strong>of</strong> <strong>the</strong> program to assess its impact will keep it relevant 

as student technology needs change over time. 

Bella Gerlich and Lynn Berard review <strong>the</strong> findings <strong>of</strong> a project that applied <strong>the</strong> 

Reference Effort Assessment Data (READ) Scale to evaluate data collected from 170 

participating reference staff at 14 academic libraries in <strong>the</strong> spring <strong>of</strong> 2007. 101 The READ 

Scale is a six-point tool used to categorize qualitative data collected during reference 

transactions. The data relates to skills, knowledge, techniques, and tools <strong>the</strong> librarians 

consulted during <strong>the</strong> reference transactions. The authors discuss practical approaches 

to employing <strong>the</strong> READ Scale data. 

Tim Bucknall compares three models for getting access to journal literature—direct 

subscriptions, pay per view, and “big deals” <strong>of</strong>fered to library consortia. 102 His analysis <strong>of</strong> 

<strong>the</strong> models reveals that both pay per view and <strong>the</strong> “big deal” <strong>of</strong>fered frequently consulted 

additional content and that both are cost effective alternatives to <strong>the</strong> traditional direct 

subscription model. Truong Luong and Chern Liew report on a project that applied a 

checklist to assess <strong>the</strong> “usability features” <strong>of</strong> <strong>the</strong> online public access catalogs (OPACs) 

<strong>of</strong> New Zealand’s academic libraries. 103 They also compare how <strong>the</strong> libraries with <strong>the</strong> 

same s<strong>of</strong>tware customized <strong>the</strong>ir interfaces to make <strong>the</strong>m more user-friendly. Most <strong>of</strong> 

<strong>the</strong> OPACs assessed received high scores on <strong>the</strong> checklist in <strong>the</strong> areas <strong>of</strong> bibliographic 

display, text, layout, labels, and assistance to <strong>the</strong> public. However new features associ- 

<strong>the</strong> assessment <strong>of</strong> <strong>the</strong> public services 


28 


ated with search engines, features such as word cloud, faceted navigation, most popular 

ranking, and related items, were not available in <strong>the</strong> OPACs. 

Liz Burke reports on a project done in 2006 that investigated <strong>the</strong> models and methods 

for delivering reference services in Australian academic libraries. Information was 

ga<strong>the</strong>red using a survey that was sent to <strong>the</strong> forty Australian university libraries. Burke 

discusses <strong>the</strong> survey’s main finding that reference services have changed in recent years 

and concludes with <strong>the</strong> observation that <strong>the</strong>re will be an expanding role for reference librarians 

in helping patrons navigate an increasingly complex information environment. 104 

Andrew Leykam reports on a study that assessed interlibrary loan (ILL) services 

at <strong>the</strong> College <strong>of</strong> Staten Island Library to determine who was using <strong>the</strong> service. ILL 

data was collected over a three-year period, and usage patterns were determined from 

<strong>the</strong> data. The researchers found that faculty members constituted <strong>the</strong> majority <strong>of</strong> <strong>the</strong> 

service’s users and that a majority <strong>of</strong> <strong>the</strong>m were from a limited number <strong>of</strong> departments. 

The author recommended that <strong>the</strong> usage patterns disclosed by <strong>the</strong> study’s data be used 

to improve liaison outreach activities to departments that seldom used ILL services. 105 

Corey Johnson and Elizabeth Lindsay explore <strong>the</strong> job components most important 

to public service librarians. They surveyed librarians at ARL libraries to discover <strong>the</strong> 

factors that determined how public service librarians spent <strong>the</strong>ir time at work. Among 

o<strong>the</strong>r things, <strong>the</strong> study disclosed opinions 

about those portions <strong>of</strong> <strong>the</strong>ir work most 

personally satisfying and about responsibilities 

for tenure and continuing education. 

Many <strong>of</strong> <strong>the</strong> librarians surveyed perceived 

Many <strong>of</strong> <strong>the</strong> librarians surveyed 

perceived a mismatch between 

what <strong>the</strong>y viewed as important 

to users and <strong>the</strong>ir own priorities 

related to achieving tenure. 

a mismatch between what <strong>the</strong>y viewed as 

important to users and <strong>the</strong>ir own priorities 

related to achieving tenure. 106 

Jan Brophy and David Bawden report 

on a study that compared Google with library databases and systems in order to assess 

<strong>the</strong>ir relative value, strengths, and weaknesses for finding information. The systems’ 

performance was assessed in terms <strong>of</strong> coverage, unique records, precision, quality, and 

accessibility <strong>of</strong> <strong>the</strong> results. The authors discuss four case studies that examined both 

quantitative and qualitative aspects <strong>of</strong> <strong>the</strong> systems. The quantitative results permitted 

assessments <strong>of</strong> “recall, precision, overlap, and similar factors.” Qualitative results 

permitted evaluation <strong>of</strong> <strong>the</strong> value <strong>of</strong> information. Among o<strong>the</strong>r things, <strong>the</strong> study’s findings 

indicated that Google had better coverage and greater accessibility, but <strong>the</strong> library 

resources provided results <strong>of</strong> higher quality. 107 

Staff Assessment Surveys 

A handful <strong>of</strong> interesting staff assessment surveys have been reported in <strong>the</strong> pr<strong>of</strong>essional 

literature. Eileen Shepherd describes <strong>the</strong> planning, implementation, and evaluation <strong>of</strong> 

a staff development program for <strong>the</strong> librarians at Rhodes University Library in South 

Africa. 108 A group <strong>of</strong> fifteen librarians participated in an eight-session course that covered 

information retrieval skills using databases and <strong>of</strong>fered an introduction to concepts 

relevant to present-day academic libraries. The content was taught using course man- 



agement s<strong>of</strong>tware in a classroom environment. Participants were required to complete a 

two-hour final test. The test results were used to determine whe<strong>the</strong>r staff were meeting 

<strong>the</strong> course’s learning outcomes and to improve <strong>the</strong> program. Elizabeth Jordan reports 

on projects undertaken by <strong>the</strong> University <strong>of</strong> Queensland Library (Australia) over an 

eight-year period beginning in 1999 to assess its staff members’ work motivation. 109 

Jordan examines <strong>the</strong> survey instrument, <strong>the</strong> methods used to analyze <strong>the</strong> data, and <strong>the</strong> 

results. Sue Baughman reports on <strong>the</strong> use <strong>of</strong> <strong>the</strong> Individual-Team-Organization (ITO) 

Survey, an assessment tool that measures library staff perceptions in a team-based 

structure. She explains that “ongoing use <strong>of</strong> <strong>the</strong> ITO Survey has helped <strong>the</strong> [University 

<strong>of</strong> Maryland] Libraries better understand <strong>the</strong> challenges <strong>of</strong> continuous learning among 

librarians and staff.” 110 

Conclusion 

In recent years, academic and research libraries have emphasized quality assessment 

and in particular student learning-outcomes assessment. This is reflected in <strong>the</strong> considerable 

number <strong>of</strong> articles, books, and conference papers about LibQUAL+ TM , <strong>the</strong> findings 

derived from application <strong>of</strong> <strong>the</strong> LibQUAL+ TM survey at various institutions, and <strong>the</strong> use 

<strong>of</strong> a variety <strong>of</strong> instruments and methods to assess student learning outcomes in information 

literacy classes and one-shot sessions. Input and output assessment has become less 

important as reflected by <strong>the</strong> fact that data from <strong>the</strong>se kinds <strong>of</strong> assessments are <strong>of</strong>ten not 

included in <strong>the</strong> annual reports <strong>of</strong> library directors. Indeed, one <strong>of</strong> <strong>the</strong> sources discussed 

in this article maintains that a substantial number <strong>of</strong> <strong>the</strong> 28 ARL libraries it studied no 

longer write an annual report. 111 However, ARL still collects such data. 

Over many years, trends occur wherein particular philosophies or methods <strong>of</strong> assessment 

become popular for a period <strong>of</strong> time during which <strong>the</strong>y get considerable use 

at institutions, are thoroughly discussed at meetings, and receive extensive coverage 

in <strong>the</strong> published literature. Total Quality Management was popular in <strong>the</strong> 1980s and 

1990s. Today’s trend emphasizes quality assessment tools like LibQUAL+ TM and student 

learning-outcomes assessment. Because <strong>of</strong> federal and state government education 

policies and <strong>the</strong> standards <strong>of</strong> <strong>the</strong> regional higher education accreditation organizations, 

quality assessment and learning-outcomes assessment will remain important and will 

likely receive extensive coverage in <strong>the</strong> literature for some time to come. As <strong>the</strong> pace <strong>of</strong> 

<strong>the</strong> digitization <strong>of</strong> journals and books increases, use statistics <strong>of</strong> <strong>the</strong>se online resources 

will become still more important. These statistics have already begun to replace <strong>the</strong> more 

traditional input and output data. 

Jon R. Hufford is Liaison Librarian to <strong>the</strong> College <strong>of</strong> Engineering in <strong>the</strong> Research, Instruction, 

and Outreach Department at Texas Tech University Library. E-mail Jon.hufford@ttu.edu . 


Notes 

1. U.S. Department <strong>of</strong> Education, A Test <strong>of</strong> Leadership: Charting <strong>the</strong> Future <strong>of</strong> U.S. Higher 

Education: A Report <strong>of</strong> <strong>the</strong> Commission Appointed by Secretary <strong>of</strong> Education Margaret Spellings 

(Washington, DC: Dept. <strong>of</strong> Education, 2006), 4, 14–15.

30 


2. Robert Dugan and Peter Hernon, “Outcomes Assessment: Not Synonymous with Inputs 

and Outputs,” Journal <strong>of</strong> Academic Librarianship 28, 6 (2002): 376–380. 

3. F. Wilfrid Lancaster, The Measurement and Evaluation <strong>of</strong> Library Services, 1 st ed. (Washington, 

DC: Information Resources Press, 1977), vii. 

4. Stephanie Wright and Lynda White, Library Assessment, SPEC Kit 303 (Washington, DC: 

Association <strong>of</strong> Research Libraries, Office <strong>of</strong> Leadership and Management Services, 2007), 

11–14. 

5. Lancaster, The Measurement and Evaluation <strong>of</strong> Library Services, 1 st ed. 

6. Sharon Baker and F. Wilfrid Lancaster, The Measurement and Evaluation <strong>of</strong> Library Services, 2 nd 

ed. (Arlington, VA: Information Resources Press, 1991). 

7. F. Wilfrid Lancaster, If You Want to Evaluate Your Library, 1 st ed. (Champaign: University <strong>of</strong> 

Illinois, Graduate School <strong>of</strong> Library and Information Science, 1988); Lancaster, If You Want 

to Evaluate Your Library, 2 nd ed. (Champaign: University <strong>of</strong> Illinois, Graduate School <strong>of</strong> 

Library and Information Science, 1993). 

8. Lancaster, If You Want to Evaluate Your Library, 2 nd ed., xv. 

9. Peter Hernon and Charles R. McClure, Evaluation and Library Decision Making, Information 

Management, Policy, and Services (Norwood, NJ: Ablex Publishing, 1990), xv, xvi. 

10. Peter Hernon et al., Statistics for Library Decision Making: A Handbook, Information 

Management, Policy, and Services (Norwood, NJ: Ablex Publishing, 1989); Peter Hernon 

and John Richardson, eds., Microcomputer S<strong>of</strong>tware for Performing Statistical Analysis: 

A Handbook Supporting Library Decision Making, Information Management, Policy, and 

Services (Norwood, NJ: Ablex Publishing, 1988). 

11. Nancy Van House, Beth Weil, and Charles McClure, Measuring Academic Library 

Performance: A Practical Approach (Chicago: American Library Association, 1990), ix. 

12. Philip Goldstein, The Future <strong>of</strong> Higher Education: A View from CHEMA—a Report from <strong>the</strong> 

Council <strong>of</strong> Higher Education Management Associations (Washington, DC: CHEMA, ECAR, 

2006), 1, 9, 17. 

13. Peter Hernon, Robert Dugan, and Candy Schwartz, Revisiting Outcomes Assessment in 

Higher Education (Westport, CT: Libraries Unlimited, 2006); Peter Hernon and Robert 

Dugan, Outcomes Assessment in Higher Education: Views and Perspectives (Westport, CT: 

Libraries Unlimited, 2004). 

14. Council for <strong>the</strong> Advancement <strong>of</strong> Standards in Higher Education, “CAS,” (2010), http:// 

www.cas.edu (accessed December 21, 2011), Glossary <strong>of</strong> Terms, CAS. 

15. Association <strong>of</strong> College and Research Libraries, The Value <strong>of</strong> Academic Libraries: A 

Comprehensive Research <strong>Review</strong> and Report (Chicago: Association <strong>of</strong> College and Research 

Libraries, 2010), http://www.acrl.ala.org/value (accessed December 21, 2011). 

16. Gail Munde and Kenneth Marks, Surviving <strong>the</strong> Future: Academic Libraries, Quality, and 

Assessment (Oxford: Chandos, 2009). 

17. Janet Hurlbert, ed., Defining Relevancy: Managing <strong>the</strong> New Academic Library (Westport, CT: 

Libraries Unlimited, 2008), ix. 

18. Joseph Mat<strong>the</strong>ws, Library Assessment in Higher Education (Westport, CT: Libraries 

Unlimited, 2007), 8; Joseph Mat<strong>the</strong>ws, The Evaluation and Measurement <strong>of</strong> Library Services 

(Westport, CT: Libraries Unlimited, 2007), xix. 

19. Wright and White, Library Assessment, SPEC Kit 303, 11–14. 

20. Council on Library and Information Resources, No Brief Candle: Reconceiving Research 

Libraries for <strong>the</strong> 21 st Century (Washington, DC: Council on Library and Information 

Resources, 2008), http://www.clir.org/pubs/abstract/pub142abst.html (accessed July 6, 

2011), v, 10–12. 

21. Fred Heath, “Library Assessment: The Way We Have Grown,” Library Quarterly 81, 1 (2011): 

7–25. 

22. Danuta Nitecki, “Space Assessment As a Venue for Defining <strong>the</strong> Academic Library,” Library 

Quarterly 81, 1 (2011): 27–59. 



23. Megan Oakleaf, “Are They Learning? Are We? Learning Outcomes and <strong>the</strong> Academic 

Library,” Library Quarterly 81, 1 (2011): 61–82. 

24. Joseph Mat<strong>the</strong>ws, “Assessing Organizational Effectiveness: The Role <strong>of</strong> Frameworks,” 

Library Quarterly 81, 1 (2011): 83–110. 

25. Stephen Town, “Value, Impact and <strong>the</strong> Transcendent Library: Progress and Pressures in 

Performance Measurement and Evaluation,” Library Quarterly 81, 1 (2011): 111–125. 

26. Gail Staines, “Towards an Assessment <strong>of</strong> Strategic Credibility in Academic Libraries,” 

Library Management 30, 3 (2009): 148–162. 

27. Martha Kyrillidou and Colleen Cook, “The Evolution <strong>of</strong> Measurement and Evaluation <strong>of</strong> 

Libraries: A Perspective from <strong>the</strong> Association <strong>of</strong> Research Libraries,” Library Trends 56, 4 

(2008): 888–909. 

28. Maria Pinto, “Developing a Checklist for Qualitative Evaluation <strong>of</strong> Service Charters in 

Academic Libraries,” Journal <strong>of</strong> Librarianship and Information Science 40, 2 (2008): 111–121. 

29. Steve Hiller and Stephanie Wright, “Turning Results into Action: Using Assessment 

Information to Improve Library Performance,” in Building Effective, Sustainable, Practical 

Assessment: Proceedings <strong>of</strong> <strong>the</strong> 2008 Library Assessment Conference, Seattle, Washington, 

4–7 August 2008, ed. S. Hiller et al., 245–252 (Washington, DC: Association <strong>of</strong> Research 

Libraries, 2009). 

30. Wright and White, Library Assessment, SPEC Kit 303. 

31. Stephanie Brasley, Penny Beile, and Irvin Katz, “Assessing Information Competence <strong>of</strong> 

Students Using iSkills TM : A Commercially Available Standardized Instrument,” 105–114, 

and Leslie Bussert, Sue Phelps, and Karen Diller, “Voices <strong>of</strong> Au<strong>the</strong>ntic Assessment: 

Stakeholder Experiences Implementing Sustainable Information Literacy Assessments,” 

165–175, both in Building Effective, Sustainable, Practical Assessment: Proceedings <strong>of</strong> <strong>the</strong> 2008 

Library Assessment Conference, Seattle, Washington, 4–7 August 2008, ed. S. Hiller et al., 

(Washington, DC: Association <strong>of</strong> Research Libraries, 2009). 

32. Larry White and Emily Blankenship, “Aligning <strong>the</strong> Assessment Process in Academic 

Libraries for Improved Demonstration and Reporting <strong>of</strong> Organizational Performance,” 

College and Undergraduate Libraries 14, 3 (2007): 107–119. 

33. Amos Lakos, “Evidence-based Library Management: The Leadership Challenge,” portal: 

Libraries and <strong>the</strong> Academy 7, 4 (2007): 431–450. 

34. Frank Allen and Mark Dickie, “Toward a Formula-based Model for Academic Library 

Funding: Statistical Significance and Implications <strong>of</strong> a Model Based upon Institutional 

Characteristics,” College and Research Libraries 68, 2 (2007): 170–181. 

35. Ronald Powell, “Evaluation Research: An Overview,” Library Trends 55, 1 (2006): 102–120. 

36. Roswitha Poll, “To Get One’s Money’s Worth: Library Management with Cost Data,” The 

Bottom Line: Managing Library Finances 19, 1 (2006): 7–11. 

37. Roswitha Poll and Philip Payne, “Impact Measures for Libraries and Information Services,” 

Library Hi Tech 24, 4 (2006): 547–562. 

38. Steve Hiller, Martha Kyrillidou, and Jim Self, “Assessment in North American Research 

Libraries: A Preliminary Report Card,” Performance Measurement and Metrics 7, 2 (2006): 

100–106. 

39. Xi Shi and Sarah Levy, “A Theory-guided Approach to Library Services Assessment,” 

College and Research Libraries 66, 3 (2005): 266–277. 

40. Association <strong>of</strong> Research Libraries, LibQual+, Charting Library Service Quality (Washington, 

DC: Association <strong>of</strong> Research Libraries, 2012) http://www.libqual.org/publications 

(accessed June 7, 2012). 

41. A. Parasuraman, L.L. Berry, and V.A. Zeithaml, “SERVQUAL: A Multiple-item Scale for 

Measuring Customer Perceptions <strong>of</strong> Service Quality,” Journal <strong>of</strong> Retailing 64, 1 (1988): 12–40. 

42. Liangzhi Yu et al., “An Epistemological Critique <strong>of</strong> Gap Theory Based Library Assessment: 

The Case <strong>of</strong> SERVQUAL,” Journal <strong>of</strong> Documentation 64, 4 (2008): 511–551; William Edgar, 

“Questioning LibQUAL+ TM : Expanding Its Assessment <strong>of</strong> Academic Library Effectiveness,” 

portal: Libraries and <strong>the</strong> Academy 6, 4 (2006): 445–465. 


32 


43. John Harer, “LibQUAL+ TM in Lilliput: Assessment Benefits for Small Academic Libraries,” 

Performance Measurement and Metrics 7, 3 (2006): 193–204. 

44. Melissa Becher and Janice Flug, “Using Student Focus Groups to Inform Library Planning 

and Marketing,” College and Undergraduate Libraries 12, 1–2 (2005): 1–18. 

45. Kathleen Miller, “Service Quality in Academic Libraries: An Analysis <strong>of</strong> LibQUAL+ TM 

Scores and Institutional Characteristics” (PhD diss., University <strong>of</strong> Central Florida, 2008), iii, 

1. 

46. Mat<strong>the</strong>w Long and Roger C. Schonfeld, Ithaka S+R Library Survey 2010: Insights from U.S. 

Academic Library Directors (New York: Ithaka, 2010), http://digitalcommons.bepress.com/ 

repository-research/39/ (accessed 11 October 2012). 

47. Ross Housewright and Roger C. Schonfeld, Ithaka’s 2006 Studies <strong>of</strong> Key Stakeholders in <strong>the</strong> 

Digital Transformation in Higher Education (New York: Ithaka, 2008), http://bibpurl.oclc. 

org/web/30385 (accessed 6 July 2011), 4. 

48. Russell Bailey and Barbara Tierney, Transforming Library Service through Information 

Commons: Case Studies for <strong>the</strong> Digital Age (Chicago: American Library Association, 2008), ix, 

xi. 

49. Richard Johnson and Judy Lu<strong>the</strong>r, The E-Only Tipping Point <strong>of</strong> Journals: What’s Ahead in 

<strong>the</strong> Print-to-Electronic Transition Zone (Washington, DC: Association <strong>of</strong> Research Libraries, 

2007), v, 1. 

50. Lloyd Davidson, “The End <strong>of</strong> Print: Digitization and Its Consequence—Revolutionary 

Changes in Scholarly and Social Communication and in Scientific Research,” International 

Journal <strong>of</strong> Toxicology 24, 1 (2005): 25–34. 

51. Shun Wong and T. Webb, “Uncovering Meaningful Correlation between Student Academic 

Performance and Library Material Usage,” College and Research Libraries 72, 4 (2011): 

361–370. 

52. Char Booth, Informing Innovation: Tracking Student Interest in Emerging Library Technologies at 

Ohio University (Chicago: Association <strong>of</strong> College and Research Libraries, 2009), 1–2. 

53. Joanna Bryant, Graham Mat<strong>the</strong>ws, and Graham Walton, “Academic Libraries and Social 

and Learning Space: A Case Study <strong>of</strong> Loughborough University Library, UK,” Journal <strong>of</strong> 

Librarianship and Information Science 41, 1 (2009): 7–18. 

54. Kathleen Murray and Inga Hsieh, “Archiving Web-published Materials: A Needs 

Assessment <strong>of</strong> Librarians, Researchers, and Content Providers,” Government Information 

Quarterly 25, 1 (2008): 66–89. 

55. Nancy Foster and Susan Gibbons, eds., Studying Students: The Undergraduate Research Project 

at <strong>the</strong> University <strong>of</strong> Rochester (Chicago: Association <strong>of</strong> Research Libraries, 2007), v–vii. 

56. Henry Pisciotta et al., “Penn State’s Visual Image User Study,” portal: Libraries and <strong>the</strong> 

Academy 5, 1 (2005): 33–58. 

57. Jeremy Shapiro and Shelley Hughes, “Information Literacy as a Liberal Art: Enlightenment 

Proposals for a New Curriculum,” Educom <strong>Review</strong> 31, 2 (1996): 31–36. 

58. American Library Association, Presidential Committee on Information Literacy, Final Report 

(Chicago: American Library Association, 1989), http://www.ala.org/ala/mgrps/divs/ 

acrl/publications/whitepapers/presidential.cfm#importance (accessed October 5, 2011). 

59. Middle States Commission on Higher Education, Student Learning Assessment: Options and 

Resources (Philadelphia: Middle States Commission on Higher Education, 2007), 1–2, 6. 

60. Dorothy Warner, A Disciplinary Blueprint for <strong>the</strong> Assessment <strong>of</strong> Information Literacy, (Westport, 

CT: Libraries Unlimited, 2008), ix, xii. 

61. Carolyn Radcliff et al., A Practical Guide to Information Literacy Assessment for Academic 

Librarians (Westport, CT: Libraries Unlimited, 2007), 3. 

62. Teresa Y. Neely, Information Literacy Assessment: Standards-Based Tools and Assignments 

(Chicago: American Library Association, 2006), 3. 

63. Joanna Burkhardt, Mary MacDonald, and Andree Ra<strong>the</strong>macher, Creating a Comprehensive 

Information Literacy Plan: A How-To-Do-It-Manual and CD-ROM (New York: Neal-Schuman, 

2005), 93–105 



64. Yu-Hui Chen and Mary Van Ullen, “Helping International Students Succeed Academically 

through Research Process and Plagiarism Workshops,” College and Research Libraries 72, 3 

(2011): 209–235. 

65. Jon Hufford, “What Are They Learning? Pre- and Post-assessment Surveys for LIBR 1100, 

Introduction to Library Research,” College and Research Libraries 71, 2 (2010): 139–158. 

66. Jon Hufford and Arlene Paschel, “Pre- and Post-assessment Surveys for <strong>the</strong> Distance 

Section <strong>of</strong> LIBR 1100, Introduction to Library Research,” Journal <strong>of</strong> Library Administration 50, 

5–6 (2010): 693–711. 

67. Brian Winterman, “Building Better Biology Undergraduates through Information Literacy 

Integration,” Issues in Science and Technology Librarianship 58 (Summer 2009), http://www. 

istl.org/09-summer/refereed1.html (accessed July 6, 2011). 

68. Megan Oakleaf, “The Information Literacy Instruction Assessment Cycle: A Guide for 

Increasing Student Learning and Improving Librarian Instructional Skills,” Journal <strong>of</strong> 

Documentation 65, 4 (2009): 539–560. 

69. Laura Saunders, “The Future <strong>of</strong> Information Literacy in Academic Libraries: A Delphi 

Study,” portal: Libraries and <strong>the</strong> Academy 9, 1 (2009): 99–114. 

70. John Budd, “Cognitive Growth, Instruction, and Student Success,” College and Research 

Libraries 69, 4 (2008): 319–330. 

71. Karen Diller and Sue Phelps, “Learning Outcomes, Portfolios, and Rubrics, Oh My! 

Au<strong>the</strong>ntic Assessment <strong>of</strong> an Information Literacy Program,” portal: Libraries and <strong>the</strong> 

Academy 8, 1 (2008): 75–89. 

72. Kate Zoellner, Sue Samson, and Samantha Hines, “Continuing Assessment <strong>of</strong> Library 

Instruction to Undergraduates: A General Education Course Survey Research Project,” 


73. Kristin H<strong>of</strong>fmann et al., “Library Research Skills: A Needs Assessment for Graduate 

Student Workshops,” Issues in Science and Technology Librarianship 53 (Winter 2008), http:// 

www.istl.org/08-winter/refereed1.html (accessed July 6, 2011). 

74. Donald Gilstrap and Jason Dupree, “Assessing Learning, Critical Reflection, and Quality 

Educational Outcomes: The Critical Incident Questionnaire,” College and Research Libraries 

69, 5 (2008): 407–426. 

75. Lana Ivanitskaya et al., “How Does a Pre-assessment <strong>of</strong> Off-campus Students’ Information 

Literacy Affect <strong>the</strong> Effectiveness <strong>of</strong> Library Instruction?,” Journal <strong>of</strong> Library Administration 

48, 3–4 (2008): 509–525. 

76. Laura Saunders, “Regional Accreditation Organizations’ Treatment <strong>of</strong> Information Literacy: 

Definitions, Collaboration, and Assessment,” The Journal <strong>of</strong> Academic Librarianship 33, 3 

(2007): 317–326. 

77. Randall Schroeder and Kimberly Mashek, “Building a Case for <strong>the</strong> Teaching Library: Using 

a Culture <strong>of</strong> Assessment to Reassure Converted Campus Partners while Persuading <strong>the</strong> 

Reluctant,” Public Services Quarterly 3, 1–2 (2007): 83–110. 

78. Lynn Cameron, Steven Wise, and Susan Lottridge, “The Development and Validation <strong>of</strong> <strong>the</strong> 

Information Literacy Test,” College and Research Libraries 68, 3 (2007): 229–236. 

79. Davida Scharf et al., “Direct Assessment <strong>of</strong> Information Literacy Using Writing Portfolios,” 

The Journal <strong>of</strong> Academic Librarianship 33, 4 (2007): 462–477. 

80. Thomas Mackey and Trudi Jacobson, “Developing an Integrated Strategy for Information 

Literacy Assessment in General Education,” The Journal <strong>of</strong> General Education 56, 2 (2007): 

93–104. 

81. Nancy O’Hanlon, “Information Literacy in <strong>the</strong> University Curriculum: Challenges for 

Outcomes Assessment,” portal: Libraries and <strong>the</strong> Academy 7, 2 (2007): 169–189. 

82. Korrielia Tancheva, Camille Andrews, and Gail Steinhart, “Library Instruction Assessment 

in Academic Libraries,” Public Services Quarterly 3, 1–2 (2007): 29–56. 

83. Gillian Gremmels and Karen Lehmann, “Assessment <strong>of</strong> Student Learning from Reference 

Service,” College and Research Libraries 68, 6 (2007): 488–501. 


34 


84. Kathrine Schilling and Rachel Applegate, “Evaluating Library Instruction: Measures for 

Assessing Educational Quality and Impact,” in Sailing Into <strong>the</strong> Future, Charting Our Destiny, 

Proceedings <strong>of</strong> <strong>the</strong> Thirteenth National Conference <strong>of</strong> <strong>the</strong> Association <strong>of</strong> College and Research 

Libraries, Baltimore, March 29–April 1, 2007, ed. H. Thompson, 206–214 (Chicago: American 

Library Association, 2007). 

85. Shikha Sharma, “From Chaos to Clarity: Using <strong>the</strong> Research Portfolio to Teach and Assess 

Information Literacy Skills,” The Journal <strong>of</strong> Academic Librarianship 33, 1 (2007): 127–135. 

86. Pamela Jackson, “Plagiarism Instruction Online: Assessing Undergraduate Students’ 

Ability to Avoid Plagiarism,” College and Research Libraries 67, 5 (2006): 418–428. 

87. Elizabeth Choinski and Michelle Emanuel, “The One-minute Paper and <strong>the</strong> One-hour 

Class: Outcomes Assessment for One-shot Library Instruction,” Reference Services <strong>Review</strong> 34, 

1 (2006): 148–155. 

88. Lorrie Knight, “Using Rubrics to Assess Information Literacy,” Reference Services <strong>Review</strong> 34, 

1 (2006): 43–55. 

89. Educational Testing Service, “ETS Launches Revised ICT Literacy Assessment,” BizEd 5, 2 

(2006): 49. 

90. Paul Hrycaj, “An Analysis <strong>of</strong> Online Syllabi for Credit-bearing Library Skills Courses,” 


91. Marlene Asselin, Margaret Early, and Margot Filipenko, “Accountability, Assessment, 

and <strong>the</strong> Literacies <strong>of</strong> Information and Communication Technologies,” Canadian Journal <strong>of</strong> 

Education 28, 4 (2005): 802–826. 

92. Barry Cull, “Voices in <strong>the</strong> Wilderness: A Report on Academic Information Literacy 

Instruction in Atlantic Canada,” The Canadian Journal <strong>of</strong> Information and Library Science 29, 1 

(2005): 1–26. 

93. Karin De Jager and Mary Nassimbeni, “Information Literacy and Quality Assurance in 

South African Higher Education Institutions,” Libri 55, 1 (2005): 31–38. 

94. Margy MacMillan, “Open Resume: Magic Words for Assessment,” College and Research 

Libraries News 66, 7 (2005): 516–520. 

95. Lori Buchanan et al., “Information Literacy in <strong>the</strong> First Year: Collaborating, Planning and 

Assessing at Austin Peay,” Tennessee Libraries 55, 2 (2005): 29–37. 

96. Tyrone Cannon, “Closing <strong>the</strong> Digital Divide: an Assessment <strong>of</strong> Urban Graduate Teacher 

Education Students’ Knowledge <strong>of</strong> Information Literacy and Their Readiness to Integrate 

Information Literacy into Their Teaching” (EdD diss., University <strong>of</strong> San Francisco, 2007), 

1–4. 

97. Virginia Williams, “Building and Evaluating Juvenile Collections in Academic Libraries,” 

College and Undergraduate Libraries 18, 1 (2011): 58–76. 

98. Robert Alan et al., “Approval Plan Pr<strong>of</strong>ile Assessment in Two Large ARL libraries: 

University <strong>of</strong> Illinois at Urbana-Champaign and Pennsylvania State University,” Library 

Resources and Technical Services 54, 2 (2010): 64–76. 

99. Mat<strong>the</strong>w Ciszek and Courtney Young, “Diversity Collection Assessment in Large 

Academic Libraries,” Collection Building 29, 4 (2010): 154–161. 

100. Jim Hahn et al., “Technology on Demand: Implementing Loanable Technology Services at 

<strong>the</strong> University <strong>of</strong> Illinois at Urbana-Champaign,” Library Hi Tech 29, 1 (2011): 34–50. 

101. Bella Gerlich and Lynn Berard, “Testing <strong>the</strong> Viability <strong>of</strong> <strong>the</strong> READ Scale (Reference Effort 

Assessment Data) © : Qualitative Statistics for Academic Reference Services,” College and 

Research Libraries 71, 2 (2010): 116–137. 

102. Tim Bucknall, “A Comparative Evaluation <strong>of</strong> Journal <strong>Literature</strong> Access Options at <strong>the</strong> 

University <strong>of</strong> North Carolina at Greensboro,” Interlending and Document Supply 37, 3 (2009): 

140–142. 

103. Truong Luong and Chern Liew, “The Evaluation <strong>of</strong> New Zealand Academic Library 

OPACs: A Checklist Approach,” The Electronic Library 27, 3 (2009): 376–393. 

104. Liz Burke, “Models <strong>of</strong> Reference Services in Australian Academic Libraries,” Journal <strong>of</strong> 

Librarianship and Information Science 40, 4 (2008): 269–286. 



105. Andrew Leykam, “Exploring Interlibrary Loan Usage Patterns and Liaison Activities: <strong>the</strong> 

Experience at a US University,” Interlending and Document Supply 36, 4 (2008): 218–224. 

106. Corey Johnson and Elizabeth Lindsay, “Why We Do What We Do: Exploring Priorities 

within Public Services Librarianship,” portal: Libraries and <strong>the</strong> Academy 6, 3 (2006): 347–369. 

107. Jan Brophy and David Bawden, “Is Google Enough? Comparison <strong>of</strong> an Internet Search 

Engine with Academic Library Resources,” Aslib proceedings: new information perspectives 57, 

6 (2005): 498–512. 

108. Eileen Shepherd, “In-service Training for Academic Librarians: A Pilot Programme for 

Staff,” The Electronic Library 28, 4 (2010): 507–524. 

109. Elizabeth Jordan, “Keeping a Finger on <strong>the</strong> Organizational Pulse: Surveying Staff 

Perceptions in Times <strong>of</strong> Change,” in Building Effective, Sustainable, Practical Assessment: 

Proceedings <strong>of</strong> <strong>the</strong> 2008 Library Assessment Conference, Seattle, Washington, 4–7 August 2008, 

ed. S. Hiller, 217–223 (Washington, DC: Association <strong>of</strong> Research Libraries, 2009). 

110. Sue Baughman, “Assessment <strong>of</strong> Teams and Teamwork in <strong>the</strong> University <strong>of</strong> Maryland 

Libraries,” portal: Libraries and <strong>the</strong> Academy 8, 3 (2008): 293–312. 

111. Gail Staines, “Towards an Assessment <strong>of</strong> Strategic Credibility in Academic Libraries,” 

148–162.

A Review of the Literature on Assessment in Academic and ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?