e-Health Readiness Assessment Tools for Healthcare - Making the ...

TELEMEDICINE AND e-HEALTH
Volume 13, Number 4, 2007
© Mary Ann Liebert, Inc.
DOI: 10.1089/tmj.2006.0064
Original Research
e-Health Readiness Assessment Tools for Healthcare
Institutions in Developing Countries
SHARIQ KHOJA, M.D., Ph.D., 1,2 RICHARD E. SCOTT, Ph.D., 1 ANN L. CASEBEER, Ph.D., 1
M. MOHSIN, M.Sc., 3 A.F.M. ISHAQ, Ph.D., 3 and SALMAN GILANI, M.B.B.S. 4
ABSTRACT
e-Health Readiness refers to the preparedness of healthcare institutions or communities for
the anticipated change brought by programs related to Information and Communications
Technology (ICT). This paper presents e-Health Readiness assessment tools developed for
healthcare institutions in developing countries. The objectives of the overall study were to
develop e-health readiness assessment tools for public and private healthcare institutions in
developing countries, and to test these tools in Pakistan. Tools were developed using participatory
action research to capture partners’ opinions, reviewing existing tools, and developing
a conceptual framework based on available literature on the determinants of access to e-
health. Separate tools were developed for managers and for healthcare providers to assess
e-health readiness within their institutions. The tools for managers and healthcare providers
contained 54 and 50 items, respectively. Each tool contained four categories of readiness. The
items in each category were distributed into sections, which either represented a determinant
of access to e-health, or an important aspect of planning. The conceptual framework, and the
validity and reliability testing of these tools are presented in separate papers. e-Health readiness
assessment tools for healthcare providers and managers have been developed for healthcare
institutions in developing countries.
INTRODUCTION
USE OF INFORMATION AND COMMUNICATION
TECHNOLOGY (ICT) in healthcare is widely
perceived as an important source of reducing
discrimination based on lack of access to information,
and as a means of timely response
to matters impacting one’s personal or community
health. 1 The idea of e-readiness in
healthcare (hereto referred as “e-health readiness”)
is relatively new, and has been defined
in a report from CANARIE Inc. as “the degree
to which users, healthcare institutions, and the
healthcare system itself, are prepared to participate
and succeed with e-health implementation.”
2 Most available literature describes e-
readiness as a response to the growing digital
divide between developed and developing
countries. 3–6 Studies also suggest that the main
intent of e-readiness assessment is to provide a
unified framework to evaluate dimensions of
the digital divide. 7 In contrast, very little liter-
1 Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada.
2 Aga Khan University, Karachi, Pakistan.
3 COMSATS Institute of Information Technology, Islamabad.
4 Holy Family Hospital, Rawalpindi, Pakistan.
425

426
ature refers to e-health readiness. Available reports
indicate e-health readiness is measured
by assessing the relative status of governments,
healthcare institutions, or users in areas most
critical for adoption and success of programs
using ICT. 2 This represents an important step
in change management, and including this step
in the planning process increases the chances
of program success as well as enhancing equity
and reducing the digital divide. e-Health readiness
assessment could also provide other advantages,
such as: (1) avoiding huge losses in
time, money, and effort; (2) avoiding delays
and disappointments among planners, staff,
and users of services; and (3) facilitating the
process of change in the institutions and communities
involved, from the stage of precontemplation
(firmness and resistance to change)
through contemplation (acceptance of new
ideas) and to preparation (preparedness for
change). 4,8 Since most of these advantages of e-
health readiness are yet to be proven, it is important
to first improve the capacity of institutions
to assess their e-health readiness. Only
then would they be able to ascertain the role
and impact of e-health readiness in the broader
implementation of e-health. Several e-readiness
assessment tools have been developed in
areas such as e-business, e-commerce, e-learning,
and e-government. 4,6,9 Most of these tools
claim to be useful both in diagnosing current
situations in terms of an organization’s or community’s
preparedness to implement ICT, and
also in orienting steps to narrow the digital divide.
7
Efforts to develop readiness assessment tools
for e-health programs have taken place in the
Canadian provinces of Alberta and Ontario
where tools to assess e-health readiness in
healthcare institutions and communities were
developed. 10,11 Despite being very comprehensive,
these tools focus on developed countries,
and do not address the determinants of reduced
ICT accessibility in developing countries.
It is, therefore, important to create tools
that are more specific to the issues faced by developing
countries. This paper describes the
process of design and construction of e-health
readiness assessment tools developed for
healthcare institutions in developing countries.
Details of validity and reliability testing in Pakistan
are provided elsewhere. 12,13
KHOJA ET AL.
MATERIALS AND METHODS
This overall study used a “mixed methods”
approach, applying a sequential exploratory
design to develop and validate e-health readiness
assessment tools. 14,15 The sequential exploratory
design is best suited for developing
and testing new instruments, particularly
when researchers look for accuracy, precision,
and application of the instrument at the same
time. According to Tashakkori, the sequential
exploratory design “is better suited to explore
a phenomenon . . . and is often discussed as a
design used when a researcher develops and
tests an instrument.” 14
In the process of using a sequential exploratory
design, the study first applied participatory
action research to gather expert opinion
on the composition and format of e-health
readiness assessment tools. The study also used
perspectives from existing e-readiness and e-
health readiness tools and the available literature.
Two e-health readiness assessment tools
were developed for application in healthcare
institutions of developing countries: one for
managers, and one for healthcare providers.
Details of the conceptual framework used for
developing the tools will be published in a separate
article.
To validate the tools, the overall study first
used a qualitative method, borrowed from the
tradition of case-study, by employing in-depth
semistructured interviews to test face and content
validity. 16,17 Details of validity testing are
published in a separate article. 12 This testing
demonstrated very good face and content validity
for both tools. To assess their reliability,
a survey methodology was used. All managers
and healthcare providers working in the participating
institutions, and having experience
with the planning or implementation of e-
health programs, were asked to complete the
e-health readiness assessment tools developed
in this study. Data were analyzed to determine
the internal consistency by calculating Cronbach’s
alpha for each tool and also for individual
categories within the tools. 18 In the final
step, both qualitative and quantitative results
were incorporated to extend the interpretation
of results. Details of reliability testing are also
published separately. 13 This testing demonstrated
very high reliability for both tools.

ASSESSMENT TOOLS FOR HEALTHCARE INSTITUTIONS 427
Ethics approval for the study was obtained
from the “Conjoint Health Research Ethics
Board” at the University of Calgary.
RESULTS
Final drafts of both tools, for managers and
healthcare providers, were approved by the
team of researchers from University of Calgary
and partner institutions in Pakistan. Based
upon the literature, existing tools, and expert
opinion, the research team agreed on both the
composition and the format of the draft e-
health readiness assessment tools.
All the items addressing a single determinant
of access to e-health or a single aspect of planning
were grouped in separate sections. There
TABLE 1.
COMPOSITION AND FORMAT OF CATEGORY OF CORE-READINESS
Statements: 1 2 3 4 5 D/K
Identification of Needs for future changes, which the proposed telehealth/
e-health project will address:
1. Organization has properly identified its needs 1 2 3 4 5 D/K
2. Organization has properly prioritized its needs 1 2 3 4 5 D/K
Dissatisfaction with status quo on the prioritized needs (related to the
proposed project):
1. There is general dissatisfaction with current handling of issues that could be 1 2 3 4 5 D/K
addressed through telehealth/e-health
2. Solutions other than telehealth/e-health have been explored. 1 2 3 4 5 D/K
Awareness about telehealth/e-health in the organization:
1. Awareness of ICT and internet’s role in healthcare exists among the planners 1 2 3 4 5 D/K
2. Awareness of ICT and internet’s role in addressing the prioritized needs 1 2 3 4 5 D/K
exists among the planners.
Comfort with technology:
1. There is general comfort in using ICT/internet among users of the proposed 1 2 3 4 5 D/K
telehealth/e-health project.
2. There is general comfort among staff in using ICT/internet for storing 1 2 3 4 5 D/K
patient information.
3. There is general comfort among staff in using ICT/internet for the purpose 1 2 3 4 5 D/K
of patient care and education.
Trust on the use of ICT:
1. All the policymakers and senior administrators trust new technology as a 1 2 3 4 5 D/K
solution to the identified problems
2. All the staff members trust new technology as a solution to the identified 1 2 3 4 5 D/K
problems
3. There are plans in place to increase staff’s trust and confidence in the new 1 2 3 4 5 D/K
technology
Planning for the new telehealth/e-health project:
1. An individual or a group has taken responsibility for planning. 1 2 3 4 5 D/K
2. All the user groups among staff and other stakeholders have been involved 1 2 3 4 5 D/K
in planning
3. There is an appropriate plan for implementation of telehealth/e-health 1 2 3 4 5 D/K
initiative
4. The implementation plan includes proper budgeting and identification of 1 2 3 4 5 D/K
resources.
5. There is an appropriate plan for evaluation of telehealth/e-health initiative, 1 2 3 4 5 D/K
including option for external evaluation
Overall satisfaction and willingness:
1. The proposed technology is appropriate according to the conditions within 1 2 3 4 5 D/K
the organization
2. There is a willingness among staff to implement the technology for its 1 2 3 4 5 D/K
intended purpose
Integration of technology:
1. Integration of technology with the current services has been considered in 1 2 3 4 5 D/K
the planning process
2. There is a plan in place to integrate telehealth/e-health with the current 1 2 3 4 5 D/K
services
Score

428
were one to four items in each section. The sections
were then grouped into four categories
for each tool; core-readiness, societal readiness,
and policy readiness were common to both the
tools. The fourth category in the tool for managers
was called “technological readiness,”
whereas the fourth category in the tool for
healthcare providers was called “learning
readiness.”
Draft tools were tested for validity and reliability.
12,13 The final tools contained 54 items
for managers and 50 items for healthcare
providers. Each of the categories, along with all
the items, is described below and illustrated in
Tables 1–5.
Category I—Core readiness
This category was common in both the tools,
and addressed the overall planning process for
a proposed e-health program, and the knowledge
and experience of planners with programs
using ICT. Specific core-readiness items
dealt with the importance of needs assessment,
key aspects of planning, and the determinants
of accessibility such as appropriateness of technology,
and integration of technology with existing
services (Table 1).
KHOJA ET AL.
Category II—Technological readiness
This category was included in the tool for managers
only, and addressed the availability and affordability
of required ICT, and the hardware
and software needed to implement a proposed
program. Specific technological readiness items
dealt with physical access to technology along
with determinants of accessibility such as affordability
and capacity building (Table 2).
Category III—Learning readiness
This category was included in the tool for
healthcare providers only, and addressed issues
related to the existence of programs and
resources to provide training to healthcare
providers in using the technology. Specific
learning readiness items dealt with the inclusion
of healthcare providers in the planning
process and determinants of accessibility such
as capacity building (Table 3).
Category IV—Societal readiness (ICT use
and interaction)
This category was included in both the tools,
and dealt with any existing interaction of the
concerned institution with other healthcare in-
TABLE 2.
COMPOSITION AND FORMAT OF CATEGORY OF TECHNOLOGICAL READINESS
Statements: 1 2 3 4 5 D/K
Speed and quality of ICT/Internet at the institution:
1. Speed of connections is appropriate for the proposed use 1 2 3 4 5 D/K
2. Quality of connections is appropriate for the proposed use 1 2 3 4 5 D/K
Service/Support for ICT:
1. Service/support is available within a reasonable time frame for the 1 2 3 4 5 D/K
proposed use.
2. Local support is proficient to address most of the problems related 1 2 3 4 5 D/K
to the proposed use.
Hardware and software:
1. Hardware and software required for the proposed project are 1 2 3 4 5 D/K
readily available.
2. Hardware and software required for the proposed project are 1 2 3 4 5 D/K
readily affordable.
Availability and affordability of the desired ICT
1. Required ICT (telephone/internet/bandwidth) is easily available 1 2 3 4 5 D/K
for the institution.
2. Required ICT (telephone/internet/bandwidth) is easily available 1 2 3 4 5 D/K
for the institutions involved.
Institutional access to ICT/Internet training:
1. Programs are in place to train the users for proposed project. 1 2 3 4 5 D/K
2. Manpower is in place to train the users for proposed project. 1 2 3 4 5 D/K
Score

ASSESSMENT TOOLS FOR HEALTHCARE INSTITUTIONS 429
TABLE 3.
COMPOSITION AND FORMAT OF CATEGORY OF LEARNING READINESS
Statements: 1 2 3 4 5 D/K
ICT/Internet training for healthcare providers:
1. Personnel and programs are in place for training 1 2 3 4 5 D/K
Use of ICT/Internet to enhance education of care providers:
1. Programs exist for continuous education 1 2 3 4 5 D/K
2. ICT/Internet is readily used in continuous education 1 2 3 4 5 D/K
3. Programs are in place to use ICT/Internet for continuous education 1 2 3 4 5 D/K
Involvement of healthcare providers in telehealth/e-health projects:
1. There is a plan in place to involve healthcare providers in the planning 1 2 3 4 5 D/K
of new telehealth/e-health interventions.
2. There is a plan in place to involve healthcare providers in the 1 2 3 4 5 D/K
implementation of new telehealth/e-health interventions.
Score
stitutions in the region and beyond. Specific societal
readiness items dealt with determinants of
accessibility such as relevance of content and sociocultural
factors, and addressed the issues of
inequity in gender and social classes (Table 4).
Category V—Policy readiness (at institutional
and government levels)
This category was also included in both the
tools, and dealt with the existence of policies
at the government and institutional levels to
address common issues such as licensing, liability,
and reimbursement. Specific policy
readiness items dealt with determinants of accessibility
such as the legal and regulatory
framework and political will (Table 5).
Scoring of e-health readiness assessment tools
Each of the categories in both the tools contained
items that determine readiness of an in-
TABLE 4.
COMPOSITION AND FORMAT OF CATEGORY OF SOCIETAL READINESS
Statements: 1 2 3 4 5 D/K
Communication with other organizations:
1. Staff regularly uses ICT/Internet to communicate with staff at the other health 1 2 3 4 5 D/K
institutions of the region
2. Staff regularly uses ICT/Internet to communicate with local community and clients 1 2 3 4 5 D/K
3. Other institutions involved in the telehealth/e-health project have also planned to 1 2 3 4 5 D/K
go through e-readiness assessment.
[For e-learning projects] Sharing of locally relevant content between healthcare
institutions:
1. Material on locally relevant health issues is shared between this institution and 1 2 3 4 5 D/K
other institutions.
2. The relevant material is available in language(s) easily understood by all the 1 2 3 4 5 D/K
concerned staff and other users of information
[For service related projects] Providing care to patients and communities in
collaboration with other healthcare institutions:
1. A referral system is available between this institution and other healthcare 1 2 3 4 5 D/K
institutions to provide patient care in certain specialities.
2. ICT is currently used for referrals between this institution and other healthcare 1 2 3 4 5 D/K
institutions.
Considering sociocultural factors among staff:
1. Both the genders have equal and unrestricted access to the technology. 1 2 3 4 5 D/K
2. Staff from all levels get direct benefit from the use of technology. 1 2 3 4 5 D/K
Considering sociocultural factors among clients and communities:
1. Use of ICT will benefit men and women equally in the society. 1 2 3 4 5 D/K
2. People from all socioeconomic strata get direct benefit from the use of technology. 1 2 3 4 5 D/K
Score

430
KHOJA ET AL.
TABLE 5.
COMPOSITION AND FORMAT OF CATEGORY OF POLICY READINESS
Statements: 1 2 3 4 5 D/K
ICT related regulations:
1. Government policies are in place to promote and manage use of telehealth/e-health 1 2 3 4 5 D/K
in healthcare institutions.
2. Institutional policies are in place to promote and manage use of telehealth/e-health 1 2 3 4 5 D/K
in your institution.
Policies regarding licensure and liability:
1. Government policies are in place to allow care provision in other jurisdictions 1 2 3 4 5 D/K
through telehealth.
2. Institutional policies are in place to allow care provision in other jurisdictions 1 2 3 4 5 D/K
through telehealth.
3. Government policies are in place to deal with liability issues. 1 2 3 4 5 D/K
4. Institutional policies are in place to deal with liability issues. 1 2 3 4 5 D/K
Policies regarding reimbursement:
1. Government policies are in place to ensure proper reimbursement to the 1 2 3 4 5 D/K
healthcare providers in your institution.
2. Institutional policies are in place to ensure proper reimbursement to the 1 2 3 4 5 D/K
healthcare providers in your institution.
Awareness and support of ICT among politicians:
1. Politicians are generally aware of the benefits of ICT use in healthcare. 1 2 3 4 5 D/K
2. Politicians generally support the use of ICT use in healthcare. 1 2 3 4 5 D/K
Awareness and support of ICT among policymakers at the institutional level:
1. Policy makers are aware of the benefits of ICT in healthcare institutions. 1 2 3 4 5 D/K
2. Policy makers support the use of ICT in healthcare institutions. 1 2 3 4 5 D/K
Score
stitution from the perspectives of managers or
healthcare providers. Each item could be rated
by the respondent using a five-point Likerttype
scale. The total score was intended to provide
longitudinal assessment that would place
an institution at different stages between prepared
and not prepared, and was not intended
for interinstitutional comparison.
DISCUSSION
In order to comprehensively assess e-health
readiness of healthcare institutions in developing
countries, the tools must address the key determinants
of accessibility to e-health relevant to
their context, along with other key planning issues
such as needs assessment, implementation,
and evaluation. Consideration of these issues
should allow greater access to e-health for staff
of different genders and levels of service, and
clients of different genders and socioeconomic
strata. Improving these issues of inequity in access
to e-health could, in turn, help in addressing
the issue of the digital divide, which is so
prominent in the developing country setting.
The e-health readiness assessment tools developed
and validated within our study are intended
for managers and healthcare providers
to use independently for the planning of e-
health programs in their own healthcare institutions.
These tools deal with factors that need
to be addressed in the planning of e-health programs,
and therefore help planners evaluate
their status and progress at various stages,
without comparison to other healthcare institutions.
Validity and reliability testing of these
tools has taken place with different levels of institutions,
sectors, and geographical regions,
making these tools more reliable and valid for
use in healthcare institutions throughout Pakistan.
The tools have also been translated into
the country’s local language (Urdu) to increase
their usability in more peripheral areas of the
country, where language can otherwise be a
significant barrier to participation and progress.
The availability of e-health readiness assessment
tools should help in improving the
quality of planning of e-health programs in
healthcare institutions of Pakistan, as well as
increasing the awareness, confidence, and trust

ASSESSMENT TOOLS FOR HEALTHCARE INSTITUTIONS 431
among staff and planners associated with e-
health applications already in use. The tools
should also help address technology and learning
issues among staff, support improved communication
between the healthcare institutions,
the clients, and the providers of care, and
also reduce barriers to the use of ICT related to
gender and socioeconomic conditions. With appropriate
contextual assessment, it is anticipated
the e-health readiness assessment tools
presented here will find broader applicability
in other developing countries, better preparing
them for introduction of e-health solutions.
ACKNOWLEDGMENTS
Funding for this article was by the Canadian
Institute for Health Research and International
Development Research Center. Partners in the
study were the COMSATS Institute of Information
Technology, Islamabad; Telemedicine
Association of Pakistan; and ProtoMed Inc.,
Pakistan.
REFERENCES
1. Tsang MW, Mok M, Kam G, Jung M, Tang A, Chan
U, Chu CM, Li I, Chan J. Improvement in diabetes
control with a monitoring system based on a handheld,
touch-screen electronic diary. J Telemed Telecare
2001;7:47–50.
2. See http://www.fp.ucalgary.ca/telehealth/Canarie%
20-%20Final%20Report,%20June%202002/Canarie
%20-%20Final%20Report,%20June%202002.pdf (Last
accessed April 30, 2006).
3. Masi CM, Suarez-Balcazar Y, Cassey MZ, Kinney L,
Piotrowski ZH. Internet access and empowerment: A
community-based health initiative. J Gen Intern Med
2003;18:525–530.
4. See www.readinessguide.org (Last accessed April 30,
2006).
5. Kalichman SC, Weinhardt L, Benotsch E, Cherry C.
Closing the digital divide in HIV/AIDS care: Development
of a theory-based intervention to increase Internet
access. AIDS Care 2002;14:523–537.
6. See www.cspp.org. (Last accessed April 30, 2006).
7. Arce ME, Hopmann C. The concept of eReadiness
and its application in developing countries: Methodological
problems and results for the eReadiness
in Nicaragua. 2002. Managua-Nicaragua. 2002.
Available at http://www.nsrc.org/CENTRAM/NI/
eReadiness-developing-countries.pdf (last accessed
December 15, 2005).
8. Prochaska JM, Prochaska JO, Levesque DA. A transtheoretical
approach to changing organizations. Administrat
Policy Mental Health 2001;28:247–261.
9. See www.bridges.org. (Last accessed April 30, 2006).
10. Jennett P, Jackson A, Healy T, Ho K, Kazanjian A,
Woolland R, Hayot S, Bates J. A study of rural community’s
readiness for telehealth. J Telemed Telecare
2003;9:259–263.
11. Flewelling C, Ingram CA. Telepediatrics in Canada:
An overview. Telemed J E-Health 2004;10:357–368.
12. Khoja S, Casebeer AL, Scott RE, Gilani SN. Validating
e-Health readiness assessment tools for developing
countries. Telemed e-Health 2006;3(1):24–30.
13. Khoja S, Scott RE, Mohsin M, Ishaq AFM. Testing reliability
of e-Health readiness assessment tools for
developing countries. e-Health Int 2006;3(1):31–37.
14. Tashakkori A. Handbook of mixed methods in social and
behavioral research. 1st ed. Thousand Oaks, CA: Sage
Publications, 2003:169–293.
15. Sandelowski M. Combining qualitative and quantitative
sampling, data collection, and analysis techniques
in mixed-method studies. Res Nurs Health
2000;23:246–255.
16. Creswell JW. Qualitative inquiry and research design:
Choosing among five traditions. 1st ed. Thousand Oaks,
CA: Sage Publications 2003;13–26.
17. Yin RK. Enhancing the quality of case studies in
health services research. Health Serv Res 1999;(5 Pt
2):1209–1224.
18. Streiner DL. A checklist for evaluating the usefulness
of rating scales. Can J Psychiatry 1993;38:140–148.
Address reprint requests to:
Shariq Khoja, M.D., Ph.D.
Department of Community Health Sciences
P.O. Box 3500
Stadium Road
The Aga Khan University
Karachi, 74800, Pakistan
E-mail: shariq.khoja@aku.edu