icegov2012 proceedings

example is online tax filing. According to the Internal Revenue
Service about 4m people filed electronically in the United States
in 1990, the first year such method was available. [5] In 2000, the
number had reached 35m. Two years ago, nearly 100m people
used e-file. One reason for the rapid rise is the fact that
government realizes its potential: “Governments are increasingly
becoming aware of the importance of employing e-government to
improve the delivery of public services to the people,” said the
2003 UN World Public Sector Report. [6]
The potential benefits of e-government extend beyond service
delivery as it can also improve accountability and performance
assuming it can increase usage and security. But measuring egovernment
across the world is difficult and because comparable
international e-government data are scarce, many prominent
benchmarks, such as those cited above, all rely on the United
Nations (UN) E-Government Development Index, in particular its
Online Service Index subset as a benchmark. However, this
assessment only captures national level initiatives at a broad level
and does not link evaluation to outcomes or impacts, i.e. egovernment
performance relative to investment.
This paper reviews the UN approach, introduces current egovernment
trends that affect measurement and explains how a
revised framework can be developed to meet these new
developments. Because of the demand for measurement at all
levels, the paper ends with a brief discussion of the revised
framework’s applicability to local level assessments.
2. THE UNITED NATIONS
E-GOVERNMENT SURVEY
2.1 Objectives
The global rise of e-government and its importance to
development broadly and the information society specifically has
been illustrated in numerous reports, but in none more so than the
biennial Global E-Government Development Reports from the
UN, published in 2003, 2004, 2005, 2008, 2010, and 2012 (plus a
2001 pilot report or a predecessor of the reports, depending on
how you view it).
The objective of the reports is to assess trends in e-government in
order to “inform and improve the understanding of policy makers’
choices to shape their e-government programs,” according to the
2004 report. [7] More recently, according to the 2010 report, the
survey measures “the willingness and capacity of countries to use
online and mobile technology in the execution of government
functions,” and compares the performance of the 193 UN member
states relative to each other. [8]
Given its worthwhile objective and the fact that comparable
international data are scarce, the UN reports have become the de
facto e-government benchmark used in numerous reports
measuring the digital society. However, relatively little attention
has been has been paid to the UN methodology. The following
section introduces the current framework based on publicly
available information.
2.2 Methodology
The E-Government Development Index is a composite index
based on three indicators, which are weighted equally. First, the
Online Service Index (OSI), which measures a country’s
availability of e-government information and services through
original research. Second, the human capital index, which is
431
derived from data primarily from the United Nations Educational,
Scientific and Cultural Organization (UNESCO) with additional
data from the United Nations Development Programme’s (UNDP)
Human Development Report, most recently the 2009 version.
Third, the telecommunications index, which is provided by the
ITU.
The human capital and telecommunications infrastructure indices
are less relevant for the purposes of this paper because they are
provided by third parties. Instead, this paper focuses on the OSI,
which is the most prominent component of the overall index and
what makes the UN assessment unique.
To measure e-government, the OSI employs a supply-side
measurement technique, which means quantitatively evaluating
websites through content analysis. It is a common approach to
measure e-government as researchers take a pre-defined survey
questionnaire and examine which information and services are
available on various websites and, to some extent, their
sophistication. For example, how many national government
websites around the world have a feature that offers to send alerts
to mobile phones? (In 2010, the answer was 25 out of 192.) Once
the entire survey questionnaire is completed, scores are simply
tallied up.
From the abridged UN methodology in the most recent report, and
the reports from the expert group meetings, a biennial exercise in
which select experts gather to discuss the UN survey and its
methodology, 1 one can deduce the following about the UN egovernment
supply-side approach: [9]
o Every UN member state, currently 193, is assessed. This
provides a broad international comparison.
o A ‘national portal’ or other official website of each UN
member state is selected for assessment, as well as associated
ministry websites (e.g., education, health, labor, social
welfare and finance). Where more than one national website
or associated portal exists, the features on relevant sites for
which links are provided are also assessed. This offers a
defined set of websites for assessment.
o A large number of e-government features and services are
assessed, each allocated to a four stage model: emerging,
enhanced, transactional and connected. In addition, eparticipation
features are assessed and the results calculated
as a separate index.
o A set of researchers visits the websites, survey in hand, and
assesses whether the features in the survey exist or not. The
team is instructed to take a citizen approach: if an item can’t
be found within a reasonable period of time, then it is
deemed not to exist (as an ordinary citizen would also give
up and instead call the government). This is important
because the UN survey is not based on whether features
actually exist but rather whether researchers can find them.
o Almost all questions in the survey call for a binary response
of yes or no, with “yes” given one point and “no” zero.” In
theory at least, this gives a quantifiable objective assessment.
Either a feature is findable or it is not, and no value
1 For the methodology, please refer to pages 119-125 of the most
recent report (UN 2012); the report from the EGM can be found
at http://www2.unpan.org/egovkb/