The Impact of Technology Insertion on Organisations

<strong>The</strong> <strong>Impact</strong> <strong>of</strong> <strong>Technology</strong>
<strong>Insertion</strong> on Organisations
<strong>The</strong> work described in this document has been undertaken by the Human Factors
Integration Defence <strong>Technology</strong> Centre, part funded by the Human Capability Domain <strong>of</strong>
the U.K. Ministry <strong>of</strong> Defence Scientific Research Programme.
© Human Factors Integration Defence <strong>Technology</strong> Centre 2007. <strong>The</strong> authors <strong>of</strong> this
report have asserted their moral rights under the Copyright, Designs and Patents act,
1988, to be identified as the authors <strong>of</strong> this work.
Reference ...............................................HFIDTC/2/12.2.1/1
Version...................................................................Version 3
Date.............................................................31 October 2007
©Human Factors Integration Defence <strong>Technology</strong> Centre 2007
.

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
Authors
Ben Dawson (HF Engineer)
ii

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
Contents
1 Executive Summary ................................................................................... 1
2 Introduction ................................................................................................ 2
2.1 Background ......................................................................................................................... 2
2.2 Research Aims and Approach ............................................................................................ 3
2.3 What is <strong>Technology</strong>?........................................................................................................... 4
2.4 Why Study <strong>Technology</strong> <strong>Insertion</strong>? ...................................................................................... 4
2.5 <strong>Impact</strong>, at What Level? ....................................................................................................... 5
2.6 Bounding and Scoping ........................................................................................................ 5
3 Studying <strong>Technology</strong>.................................................................................. 7
3.1 <strong>The</strong> History <strong>of</strong> <strong>Technology</strong> .................................................................................................. 7
3.1.1 <strong>The</strong> Industrial Revolution .......................................................................................... 7
3.1.2 <strong>The</strong> IT Revolution...................................................................................................... 7
3.2 Reasons for the Introduction <strong>of</strong> New <strong>Technology</strong>............................................................... 8
4 <strong>The</strong> Problem <strong>of</strong> <strong>Technology</strong> <strong>Insertion</strong>....................................................... 10
4.1 A Background <strong>of</strong> High Risk ............................................................................................... 10
5 Risks to Successful <strong>Technology</strong> <strong>Insertion</strong> ................................................ 12
5.1 Background ....................................................................................................................... 12
5.2 Technical Risks .................................................................................................................12
5.2.1 Project Size............................................................................................................. 13
5.2.2 Maturity <strong>of</strong> <strong>Technology</strong>............................................................................................ 14
5.2.3 Human Centred Design .......................................................................................... 14
5.2.4 Interoperability <strong>of</strong> Components............................................................................... 15
5.3 Organisational Risks ......................................................................................................... 19
5.3.1 Organisational Barriers to <strong>Technology</strong> Changes.................................................... 20
6 <strong>Impact</strong>s <strong>of</strong> <strong>Technology</strong> on the Organisation ............................................. 26
6.1 Performance and Productivity........................................................................................... 26
6.1.1 Usability................................................................................................................... 27
iii

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
6.2 Manpower Levels and Organisational Size ...................................................................... 28
6.3 Distribution <strong>of</strong> Power and Control ..................................................................................... 29
6.4 Job Enrichment vs Deskilling ............................................................................................ 29
6.5 Information Management and Policy ................................................................................ 31
6.6 <strong>Impact</strong> at the Individual and Group Level ......................................................................... 31
7 <strong>The</strong>ories <strong>of</strong> <strong>Technology</strong> <strong>Impact</strong> ................................................................ 33
7.1 <strong>The</strong>ories <strong>of</strong> <strong>Technology</strong> in Organisations ......................................................................... 33
7.1.1 Optimists and Pessimists........................................................................................ 33
7.1.2 <strong>Technology</strong> Led <strong>The</strong>ories ....................................................................................... 34
7.1.3 Strategic Choice...................................................................................................... 35
7.1.4 Integrationist Models............................................................................................... 35
7.1.5 Chaos <strong>The</strong>ory and Complexity <strong>The</strong>ory ................................................................... 37
8 <strong>The</strong> Introduction <strong>of</strong> <strong>Technology</strong>................................................................ 39
8.1 <strong>The</strong> Reality <strong>of</strong> <strong>Technology</strong> <strong>Insertion</strong> in Organisations ...................................................... 39
8.1.1 Method <strong>of</strong> Implementation ...................................................................................... 40
8.2 Risk Assessment in <strong>Technology</strong> Programmes ................................................................. 41
8.2.1 Risk Assessment Models........................................................................................ 42
8.2.2 Knowledge Management ........................................................................................ 43
9 <strong>Technology</strong> <strong>Insertion</strong> Case Studies.......................................................... 48
9.1 Military ............................................................................................................................... 48
9.1.1 Eur<strong>of</strong>ighter Typhoon ............................................................................................... 48
9.1.2 Armed Forces Health Longitudinal <strong>Technology</strong> Application................................... 49
9.2 Civil.................................................................................................................................... 50
9.2.1 <strong>The</strong> Regional Information Systems Plan ................................................................ 50
9.2.2 London Ambulance Service Computer Aided Despatch ........................................ 51
9.3 Discussion <strong>of</strong> Case Studies .............................................................................................. 53
10 Managing <strong>Technology</strong> <strong>Insertion</strong> ............................................................... 54
10.1 Methods <strong>of</strong> Supporting <strong>Technology</strong> <strong>Insertion</strong> ................................................................... 54
10.1.1 Open Systems Task Analysis ................................................................................. 54
10.1.2 Personnel and Organisational Implications <strong>of</strong> New <strong>Technology</strong> ............................ 54
10.1.3 Discussion <strong>of</strong> OSTA and POINTS Methods ........................................................... 55
iv

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
10.2 Tools and S<strong>of</strong>tware to Support Practitioners .................................................................... 55
10.2.1 Supporting <strong>Technology</strong> <strong>Insertion</strong> Programmes ...................................................... 55
10.3 Summary <strong>of</strong> the Extant TI Tools........................................................................................ 60
11 Gaps in <strong>Technology</strong> <strong>Insertion</strong> Capabilities ............................................... 62
11.1 Gaps in Understanding, Methods and Tools .................................................................... 62
11.2 Identifying and Training the <strong>Technology</strong> <strong>Insertion</strong> Practitioners....................................... 63
11.3 Are there any common lessons? ...................................................................................... 63
11.4 Can we predict whether TI will succeed or fail?................................................................ 64
11.5 A Toolkit to Support Practitioners ..................................................................................... 64
11.5.1 Identifying Risks...................................................................................................... 64
11.5.2 Decision Support..................................................................................................... 65
12 Direction <strong>of</strong> Future Work .......................................................................... 66
12.1 Summary and Follow on ................................................................................................... 66
13 References............................................................................................... 67
v

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
1 Executive Summary
<strong>Technology</strong> <strong>Insertion</strong>, the activity <strong>of</strong> introducing new technology into an existing system
is a massive challenge and one that is receiving more attention from technology
researchers and practitioners. It is also clear that the development <strong>of</strong> technology has a
long history <strong>of</strong> impacting upon the organisation. This report focuses on the challenges <strong>of</strong>
technology insertion as well as the impacts that technologies have had upon organisations
during recent decades.
A literature review was carried out in order to gain insight into the existing knowledge
surrounding the areas <strong>of</strong> technology insertion and the impact <strong>of</strong> technology upon
organisations. Case studies were selected from both the extant literature and from
knowledge gathered during interviews with subject matter experts.
<strong>The</strong> literature suggests that technology insertion projects, and technology projects as a
whole, are prone to failure. It appears that only around 25 to 50 per cent <strong>of</strong> projects
successfully integrate new technology with the business goals <strong>of</strong> the organisation. <strong>The</strong>
reasons for such a high failure rate are the number and type <strong>of</strong> risks that threaten
technology programmes. Research has found that the implementation <strong>of</strong> new technology
is a stressful experience, and can be the most time-consuming, expensive and frustrating
part <strong>of</strong> any technology project. Despite the development <strong>of</strong> risk assessment tools and
techniques, research has found that the rate <strong>of</strong> technology project failure has not reduced.
<strong>The</strong> case study evidence suggests that risk assessment tools are applied to these projects,
but they do not support assessors in identifying either common or unexpected risks.
When looking at the impact <strong>of</strong> technology on organisations, the early literature assumes
technology to be an objective, external force that has a deterministic impact on
organisational properties such as size and structure. In contrast, later researchers focused
on the human intentions associated with bringing in technology. More recent thinking
recognises the contribution <strong>of</strong> both technological and social factors on organisational
outcomes, an approach developed in response to the perceived deficiencies <strong>of</strong>
technological and social determinist positions.
Current thinking suggests that it is not possible to predict outright whether a technology
will be a success or failure for the simple reason that there are too many interacting
organisational and technical variables at work. This evidence means that it is currently
very difficult, if not impossible, to accurately predict the impact that a new technology
will have upon an organisation.
This scoping study argues that existing risk assessment and project management tools do
not provide sufficient support to technology insertion practitioners. Existing tools allow
users to, inadvertently, take a narrow view and ignore key organisational, contextual and
systemic factors. A toolkit designed to alert the practitioner to a broader range <strong>of</strong> issues,
including the technical, human, systemic and organisational, would be a valuable
development in helping achieve successful technology insertion programmes.
1

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
2 Introduction
This report was produced by the Human Factors Integration Defence <strong>Technology</strong> Centre
(HFI DTC) on behalf <strong>of</strong> the Research Acquisition Organisation (RAO). <strong>The</strong> Ministry <strong>of</strong>
Defence (MoD) is currently concerned about the impact <strong>of</strong> introducing new technology
within military organisations. As one <strong>of</strong> the key HFI DTC output owners, the Directorate
<strong>of</strong> Analysis, Experimentation and Simulation (DAES) requested that research be
conducted to examine the issues surrounding technology insertion and the impact
introduction <strong>of</strong> technology has upon organisations.
‘<strong>The</strong> levels <strong>of</strong> sophistication <strong>of</strong> emerging equipments and concepts for the Armed Forces
have the capability to <strong>of</strong>fer significant benefits to the performance <strong>of</strong> military operations
at all levels in the armed forces. <strong>The</strong> likely benefits that will be accrued from these
equipments and concepts range from achievement <strong>of</strong> more timely and appropriate effects
to effecting manpower savings and workload reductions. However, the insertion <strong>of</strong> new
technology, or introduction <strong>of</strong> new concepts on to existing structures will almost certainly
reveal socio-technological threats that could inhibit or negate the anticipated benefits’
[DAES/Dstl Task Requirement Document to HFI DTC].
<strong>The</strong> aim <strong>of</strong> this phase <strong>of</strong> the work is to provide an in-depth review <strong>of</strong> the state <strong>of</strong> the art
research into this topic and investigate the feasibility <strong>of</strong> developing a tool kit that could
be used to identify the likely impacts <strong>of</strong> emerging technologies and concepts on
organisations across all Defence Lines <strong>of</strong> Development (DLOD).
2.1 Background
<strong>The</strong> evidence suggests that besides the intended consequences <strong>of</strong> bringing in a new
technology, other, unplanned organisational impacts may occur. This study examines
whether these impacts are predictable or boundable to any extent.
Inserting new technologies into established systems and organisations is a massive
challenge, and one that is receiving more attention from technology researchers and
practitioners. This report details the risks that lead to <strong>Technology</strong> <strong>Insertion</strong> (TI) failures
and describes the methods and techniques currently available to practitioners and
planners.
This scoping study seeks to understand why technology insertion is so problematic and
whether tools or techniques can be developed to support those whose responsibility it is
to undertake TI projects.
<strong>The</strong> usage <strong>of</strong> the term <strong>Technology</strong> <strong>Insertion</strong> in many definitions relates to the
introduction <strong>of</strong> new technologies into existing systems that are at stages equivalent to the
UK’s post- Main Gate. <strong>The</strong> term ‘technology insertion’ appears to be a military one and
has arguably emerged as a direct consequence <strong>of</strong> the difficulties military organisations
have experienced whilst upgrading platform components and in attempting to introduce
new technologies into service.
According to QinetiQ’s TIMPA (<strong>Technology</strong> <strong>Insertion</strong> Major Project Area) website [1],
<strong>Technology</strong> <strong>Insertion</strong> is:
2

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
‘the activity <strong>of</strong> introducing new technology, such as a new, more powerful radio, into an
existing system, such as an aircraft.’
Despite the fact that some definitions <strong>of</strong> technology insertion refer to the introduction <strong>of</strong>
new equipment on to established systems, this scoping study considers all aspects <strong>of</strong> TI
including the introduction <strong>of</strong> wholly new or replacement systems. This approach was
taken to ensure that valuable knowledge, methods and experience from civil and
commercial industries, where project timescales and product lifecycles are <strong>of</strong>ten much
shorter, were not omitted.
2.2 Research Aims and Approach
A literature review was carried out in order to gain insight into the extant knowledge
surrounding the areas <strong>of</strong> technology insertion and the impact <strong>of</strong> technology upon
organisations. Sources <strong>of</strong> evidence included searches <strong>of</strong> open literature and knowledge
searches by the Defence Science and <strong>Technology</strong> Laboratory (DSTL). Searches included
the use <strong>of</strong> keyword terms such as ‘technology’, ‘impact’, ‘technology insertion’,
‘organisational impact <strong>of</strong> technology’, ‘technology integration’ ‘socio-technical
research’, ‘technology failure’ and ‘technology success’ amongst others. Case studies
were selected from both the literature and from knowledge gathered during interviews
with subject matter experts.
Interviewees included:
• A Pr<strong>of</strong>essor from Loughborough University & <strong>The</strong> Bayswater Institute. who
has expertise in the areas <strong>of</strong> human, social and organisational implications <strong>of</strong>
technological change (especially information technology); and the development <strong>of</strong>
techniques (e.g. task analysis, job design, responsibility analysis, s<strong>of</strong>tware
interface specification) for use in socio-technical systems design was particularly
relevant to this area <strong>of</strong> research.
• Staff member, from DSTL Portsdown. Whose involvement in the development
<strong>of</strong> DSTL’s Socio-technical Team-working for Organisation Relational Modelling
<strong>of</strong> team behaviour (STORM model) meant they had valuable experience <strong>of</strong>
modelling complex organisational interactions. <strong>The</strong> technical requirements
document specified that this scoping study should take account <strong>of</strong> this work, and
as will be discussed later, collaboration with DSTL may be valuable in later
phases.
• SO1 Human Systems was able to provide insight into how new technologies
impact the army and how new systems can affect changes to job characteristics as
well as overall military capability.
• Director <strong>of</strong> Equipment Capability (DEC) Ground Manoeuvres (GM). Whose
interest and experience in the area <strong>of</strong> technology insertion projects made them a
valuable contributor. We were able to gain particular insight into how technology
insertion and component upgrades can affect overall vehicle (and system)
performance. <strong>The</strong> DEC perspective on procurement, acquisition and the MoD UK
procurement policy <strong>of</strong> using the CADMID cycle (Concept Assessment
Development Manufacturing In-Service Disposal) was also valuable in guiding
our understanding <strong>of</strong> the key issues affecting the MoD.
3

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
• Technologist Adviser to HFI. Whose considerable experience <strong>of</strong> aerospace
engineering programmes provided valuable case study material and examples <strong>of</strong>
how seemingly insignificant component choices in technology insertion
programmes can lead to vastly different outcomes. <strong>The</strong> importance <strong>of</strong> nontechnical
as well as technical evaluation <strong>of</strong> equipment choices was a valuable
insight into technology insertion issues.
2.3 What is <strong>Technology</strong>?
<strong>The</strong> Oxford English Dictionary defines “technology” as the application <strong>of</strong> scientific
knowledge for practical purposes. <strong>The</strong> word has origins from the Greek word
‘tekhnologia’ meaning ‘systematic treatment’. In terms <strong>of</strong> its modern usage, technology is
a broad term referring to the use and knowledge <strong>of</strong> humanity’s tools and crafts.
It is very difficult to obtain a useful definition <strong>of</strong> technology. According to the fields <strong>of</strong>
science and engineering, where technology is developed there are many kinds <strong>of</strong>
technologies. Generally, the following distinctions can be made:
Science is the formal process <strong>of</strong> investigating natural phenomena. It produces information
and knowledge about the world.
Engineering is the goal-oriented process <strong>of</strong> designing and building tools and systems to
exploit natural phenomena for a practical human means. Engineers work within the
constraints <strong>of</strong> natural laws and societal needs to create technology.
<strong>Technology</strong> is the consequence <strong>of</strong> these two processes and societal requests. Most
commonly, the term technology is used as the name <strong>of</strong> all engineering products.
<strong>The</strong> conceptualisation <strong>of</strong> technology is an ongoing debate amongst technological
theorists, who ask questions such as ‘is technology more than just hardware?’ However
focussing on this debate, which quickly becomes a philosophical one, would only be a
distraction from the matter at hand. This report will, consequently, consider technology to
include engineered products, including equipment, components, hardware, s<strong>of</strong>tware,
information technologies and information systems.
<strong>The</strong> primary focus <strong>of</strong> this study will be upon the following technology categories, as they
are perceived to be the most relevant to technology insertion in modern civil and military
organisations:
• Hardware, Equipment, Components and, Information <strong>Technology</strong>, including
computer hardware, s<strong>of</strong>tware, information systems and communications
<strong>The</strong>se categories take machines and devices into account, as well as social structures,
command, control, and infrastructures.
2.4 Why Study <strong>Technology</strong> <strong>Insertion</strong>?
<strong>The</strong> introduction <strong>of</strong> new technology into military platforms is <strong>of</strong>ten very appealing, as it
<strong>of</strong>fers the potential to improve capability, whilst simultaneously reducing organisational
costs and manpower requirements. Despite this optimism, it has become apparent that the
successful introduction <strong>of</strong> new technology is by no means straightforward. As this report
will go on to describe, the introduction <strong>of</strong> new technology and the change to an
4

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
established work system has considerable potential to disrupt an already efficient
performance rather than enhance it [2].
In the military domain technology insertion problems can lead to severe consequences for
capability, performance and ultimately the survivability <strong>of</strong> personnel in the battlespace
and as a result is undoubtedly a topic worthy <strong>of</strong> further consideration.
2.5 <strong>Impact</strong>, at What Level?
This report deliberately focuses on the impact <strong>of</strong> technology upon organisations, the
employees and personnel who make up the organisation and upon the end users <strong>of</strong> new
technology systems.
2.6 Bounding and Scoping
<strong>The</strong> original brief, which outlined the technical requirements for this report was titled
‘Organisational <strong>Impact</strong> <strong>of</strong> <strong>Technology</strong> <strong>Insertion</strong>’ and went on to describe a concern that
the insertion <strong>of</strong> ‘new technology, or … new concepts onto existing structures will almost
certainly reveal socio-technical threats that could inhibit or negate the anticipated
benefits’.
Interestingly the use <strong>of</strong> the term technology insertion is not mentioned beyond the title
and the first paragraph <strong>of</strong> the technical requirement. Instead questions on the
‘organisational implications and aspects <strong>of</strong> the introduction <strong>of</strong> new technology and
concepts’ were emphasised. It was this emphasis on ‘organisational impacts’ <strong>of</strong> new
technology led the author to broaden the focus <strong>of</strong> the work beyond issues exclusively
related to ‘technology insertion’ or ‘mid life upgrade’ on existing systems or platforms.
This allowed the customers’ queries regarding the impacts <strong>of</strong> ‘new technology’ to be
addressed. Broadening the review provided an opportunity to explore research from nonmilitary
and commercial domains where there is also long experience <strong>of</strong> managing the
impact <strong>of</strong> new technology on organisations.
Some <strong>of</strong> the key topics that the author was asked in the research brief included:
• ‘How have technologies and concepts impacted on organisations historically?’
• ‘How have emergent properties been identified following organisational
perturbations?
• ‘What has been the impact <strong>of</strong> these emergent properties on organisational
behaviour?’
Having established the state <strong>of</strong> the art on the topic, further activity was to investigate the
feasibility <strong>of</strong> developing a toolkit for identifying the likely impact <strong>of</strong> inserting new
technology into organisations.
<strong>The</strong> research questions addressed in this study were, by their nature, incredibly broad.
<strong>The</strong> aim <strong>of</strong> this study was to try to scope the breadth <strong>of</strong> the problem area and to identify
those avenues that represented most value from the investment <strong>of</strong> further research effort.
In order to investigate ‘socio technical threats’, ‘organisational perturbations’ and the
‘impact <strong>of</strong> technologies on organisations’ the review <strong>of</strong> literature concentrated on
5

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
disciplines such as socio-technical systems, human factors, risk management,
organisational science, change management, which specifically seek to address such
areas. Harder engineering topics such as ‘system architecture’ and ‘modular design’ were
not explored in this study because <strong>of</strong> the focus <strong>of</strong> the research. <strong>The</strong> specific issue <strong>of</strong>
emerging technologies was not considered because their impact upon organisations had
not yet occurred, nor, as history suggests, could their impacts be easily predicted.
However, readers interested in journalistic comment on technology trends and
speculation on the impacts <strong>of</strong> emerging technologies may want to take a look at the MIT
journal ‘technology review’.
<strong>The</strong> remainder <strong>of</strong> this report describes the key findings from the literature and interviews
associated with TI including: risks in technology projects, human factors implications,
and theoretical explanations for organisational impacts. Tools and techniques available to
support management <strong>of</strong> TI are explored, and real-world case studies are reviewed from
military and civil domains to illustrate where and how problems can arise. <strong>The</strong> final
section makes recommendations for development <strong>of</strong> tools to support managing TI in
organisations.
6

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
3.1 <strong>The</strong> History <strong>of</strong> <strong>Technology</strong>
3.1.1 <strong>The</strong> Industrial Revolution
3 Studying <strong>Technology</strong>
When looking at the impact <strong>of</strong> technology upon organisations it is possible to look as far
back as the industrial revolution for useful lessons to be learned. <strong>The</strong> birth <strong>of</strong> industry and
the introduction <strong>of</strong> machine technology into the workplace meant that working people
found increased opportunities for employment in the new mills and factories. <strong>The</strong>
consequences for the workforce were strict working conditions with long hours <strong>of</strong> labour
dominated by a pace set by machines. <strong>The</strong> impact <strong>of</strong> technology on organisations was a
complete restructuring <strong>of</strong> the labour force and a massive increase in productivity and
output. A further consequence <strong>of</strong> the new technology <strong>of</strong> the Industrial Revolution was
that concentrated labour in mills, factories and mines facilitated the organisation <strong>of</strong> trade
unions, which advanced the interests <strong>of</strong> working people. An adjacent organisational
impact was a significant increase in worker wages during the period 1813-1913 [3].
<strong>The</strong> current study does not focus as far back in history as the industrial revolution, but it
is clear that the development <strong>of</strong> technology has a long history <strong>of</strong> impacting upon the
organisation. Instead ‘new technology’, as Arnold, Robertson and Cooper [4] put it:
‘usually refers to a particular set <strong>of</strong> changes that have occurred from the 1970’s
onwards… brought on by the development <strong>of</strong> microchips.’
This investigation focuses on the changes and impacts that technologies have had upon
organisations during recent decades.
3.1.2 <strong>The</strong> IT Revolution
Microchips have made it possible to build complex electronic systems cheaply and in
only a tiny fraction <strong>of</strong> the weight and size that would formerly have been required [5].
Although microchips have been around for some time now, what can be done with them
continues to advance rapidly, producing ‘a continuing high rate <strong>of</strong> technological change
in the workplace’ [6].
<strong>The</strong> literature shows what we know to be self evident, that investment in computers has
increased steadily and dramatically since at least 1971. <strong>The</strong>re was a tenfold increase in
Information <strong>Technology</strong> (IT) investments between 1971 and 1990 [7]. Despite the fallout
from the ‘dotcom’ crash in 2003 each <strong>of</strong> the major business sectors has shown the same
accelerating trend toward an increased use <strong>of</strong> IT.
Recently, ‘Moore’s Law’, a prediction that there will be a doubling <strong>of</strong> computing power
every 18 months has celebrated its 40 th anniversary and continues to be a surprisingly
accurate prediction [8].
Without microchips, which are continually improving their performance and shrinking in
size, many technologies which impact upon industry, military and modern life would not
be as sophisticated. Computer terminals would not be able to process data as quickly,
Network Enabled Capability (NEC) technologies would not be feasible and the remote
7

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
and automated piloting <strong>of</strong> vehicles such as military aircraft would still be in the realms <strong>of</strong>
science fiction.
3.2 Reasons for the Introduction <strong>of</strong> New <strong>Technology</strong>
<strong>The</strong>re are a number <strong>of</strong> reasons why organisations decide to introduce new technology.
Some <strong>of</strong> the reasons identified in the literature are:
• to reduce costs;
• to increase productivity;
• to increase quality;
• to reduce dependence on skilled labour;
• because it always seems a good idea to be up to date;
• because competitor organisations are also introducing new technology;
• because new technology is interesting;
• in order to change the relations between various groups in the organisation.
Clearly those responsible for justifying new technology expenditure may not admit to
some <strong>of</strong> these, but nonetheless these reasons have been identified in research [4].
Another reason that is <strong>of</strong>ten given for continued investment in new technology is the
exponential decline in both the price and performance <strong>of</strong> computers and technology [9].
In recent years, the development <strong>of</strong> the internet and the shift towards globalisation <strong>of</strong>
economies and industries has served to further drive the organisational investment in
technology. Ives & Jarvenpaa [10] found that the Information Systems (IS) related
literature <strong>of</strong>ten recommends global organisations to utilise IT for increasing control and
co-ordination <strong>of</strong> their business operations and in order to enable access to new global
markets and businesses. It is arguments such as these that have contributed to the
continued organisational investment in new technologies.
Bartlett and Ghoshal [11] claim that ‘firms operating in global markets will increasingly
be at a serious strategic disadvantage it they are unable to firmly control their worldwide
operations and manage them in a globally co-ordinated manner.’ Within this model
corporations are focusing on more close co-ordination <strong>of</strong> increasingly complex and global
processes and are using technology as the means <strong>of</strong> management and control.
<strong>The</strong> above findings relate to civil and commercial investment in technology. However, it
is clear that the reasons driving the introduction <strong>of</strong> new technology are very similar in the
military domain. In a recent RAO report [12], stakeholders interviewed describe the
following as key drivers <strong>of</strong> technology insertion:
1. Cost reduction
‘Cost reduction was perceived to be a major driver <strong>of</strong> TI. Key factors include cost/benefit
analyses, cheaper upgrades, a preference for COTS (Commercial Off <strong>The</strong> Shelf) over
8

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
bespoke systems, reduction in overheads, civilian spin-ins, streamlining headquarters
(HQs) and decision making, and NEC.’
2. Military capability requirements
‘In response to meeting capability requirements the underlying factors were named as
developing more effective solutions, introducing new technology, responding to
battlespace transformation requirements (i.e. scale, intensity, timeliness <strong>of</strong> response,
coalition interoperability), depth to which technology meets requirement, undertaking
new tasks with legacy systems, speed at which technology is needed and system
obsolescence.’
3. Procurement
A number <strong>of</strong> procurement factors were given as drivers <strong>of</strong> technology insertion. Although
the RAO report does not fully explain how these aspects are seen to drive technology
insertion, aspects such as ‘industrial capacity and sources <strong>of</strong> supply; manufacturing
readiness for production; advances in manufacturing technology, interoperability and
integration with other systems’ are given. This procurement driver <strong>of</strong> technology seems
similar to the forces <strong>of</strong> ‘technology push’. New technologies, providing reduced supply
and manufacturing costs that <strong>of</strong>fer increased capability, are attractive to those involved in
the procurement <strong>of</strong> technologies and equipment.
4. Knowledge
‘Maintaining the MoD knowledge base’ is another important driver for TI according to
the RAO’s report. A thorough understanding <strong>of</strong> the impact <strong>of</strong> new technology is seen to
be essential if the UK is to maintain its position on the world stage [12]. This can be
realised by noting the importance <strong>of</strong> the academic scientific evidence base; regularly
reviewing advances in underpinning technology; keeping pace with current developments
and appreciating the opportunities TI <strong>of</strong>fers.
5. Legislation
<strong>The</strong> introduction <strong>of</strong> new equipment and technology in the UK ‘is restrained by laws and
regulations and the MoD issues further restraints’. Legislation such as ‘the control <strong>of</strong>
vibration at work regulations’, amongst others, is seen as a driver for technology insertion
programmes. <strong>The</strong>se programmes are undertaken with the aim <strong>of</strong> ‘ensuring that existing
platforms and new systems meet new and existing legislation on issues such as health and
safety, protection and duty <strong>of</strong> care.’
<strong>The</strong> above list suggests that individuals within the MoD have identified similar reasons
for the introduction <strong>of</strong> technologies as have been found in the literature. Indeed these
views suggest that ‘technology push’ is a key driver and that reduced costs and increased
capabilities <strong>of</strong> new technologies prove attractive to procurers. That the same drivers exist
within the MoD as other organisations is unsurprising, however, as will be discussed later
in this report, procurers should be wary <strong>of</strong> inserting new technologies as a result <strong>of</strong> the
forces <strong>of</strong> ‘technology push’. <strong>Technology</strong> push can cause problems, as by definition it
involves technology development that is driven by ideas or capabilities in the absence <strong>of</strong>
any specific customer need. As well as this, in many instances, the promised capability
increases and cost reductions <strong>of</strong> new technologies do not emerge. <strong>The</strong> reasons for this
will be discussed in the following section <strong>of</strong> this report.
9

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
4 <strong>The</strong> Problem <strong>of</strong> <strong>Technology</strong> <strong>Insertion</strong>
4.1 A Background <strong>of</strong> High Risk
<strong>Technology</strong> insertion projects and technology projects as a whole are notoriously prone
to failure. Clegg et al [13] estimate that only 25 to 50 per cent <strong>of</strong> projects successfully
integrate new technology with the business goals <strong>of</strong> the organisation. Moynihan [14]
describes the implementation <strong>of</strong> an Information <strong>Technology</strong> (IT) system as a stressful
experience, 'it can be the most time consuming, expensive and frustrating part <strong>of</strong> any IT
project'. Similarly, Lyytinen and Hirschheim [15] found that more than 50 per cent <strong>of</strong>
technology system development projects are either partial or complete failures.
Hochstrasser and Griffiths [16] found that risk levels are ‘heightened when working with
new technology as effects on timing, costs and delivery deadlines are exacerbated’.
<strong>The</strong>re are numerous different types <strong>of</strong> technology, and technology insertion challenges, as
well as failures, are by no means limited to IT projects. In fact the evidence suggests
similar patterns and rates <strong>of</strong> failure occur across many types <strong>of</strong> technologies, from
computers to telecoms, materials, transport and bio technologies to name a few. <strong>The</strong>
emphasis on IT, especially in this section <strong>of</strong> the report, is simply a consequence <strong>of</strong> the
volume, availability and quality <strong>of</strong> research that has been published in the area <strong>of</strong> IT
failure. <strong>The</strong> wide-ranging experience that people have with IT failure (itself a
consequence) would appear to be a good place to start with this investigation.
Looking at IT failure more specifically, a 1998 review <strong>of</strong> 100 failed IT projects revealed
that 87 per cent exceeded their budgets by more than 50 per cent while 45 per cent <strong>of</strong> the
projects failed to produce the expected results [17], [17]. In fact it is not unusual for IT
projects to be abandoned before a product is delivered [19]. Sometimes IT artefacts are
not used [20] and there is evidence to suggest that there is a greater risk <strong>of</strong> failure in IT
projects than any other aspect <strong>of</strong> business [21], [22], [23]. Indeed a range <strong>of</strong> studies show
that the IT component adds a different dimension <strong>of</strong> risk which ‘all too <strong>of</strong>ten can tip the
balance towards project failure, rather than towards project success’.
Interestingly, amongst all <strong>of</strong> the early predictions <strong>of</strong> technology’s impact on
organisations, the high level <strong>of</strong> system failure was not widely predicted [24]. Consistent
figures on the rate <strong>of</strong> failure are hard to come by, with some, such as Gibbs [25] reporting
that the level <strong>of</strong> operational failure is up at 70 per cent. However, in a review <strong>of</strong> the
literature Eason [24] states that:
‘<strong>The</strong> rate started at around 40 % and, despite vast improvements in the technology, has
stubbornly refused to decrease through the many surveys conducted in the past 30 years.’
As Eason also states, these figures hide many variations in success <strong>of</strong> different types <strong>of</strong>
application but one safe general conclusion can be drawn; the bigger and more expensive
the project, the more likely it is to fail.
As well as outright failure, the evidence points to high risk <strong>of</strong> deadline and budget
overruns in technology programmes. Many IT projects, for example, greatly exceed their
budgets and planned development schedules [26]. Similarly, a study performed in 1995
[27] found that only 26 per cent <strong>of</strong> all Information Systems projects are completed on
time and within budget, with all requirements fulfilled. Research by Legris, Ingham and
Collerette [28] found that 46 per cent <strong>of</strong> technology projects were over budget, late, and
10

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
with fewer features and functions than originally specified. <strong>The</strong> statistics demonstrate a
significant and chronic failure in the scheduling and budgeting <strong>of</strong> IT and other
technology projects.
11

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
5 Risks to Successful <strong>Technology</strong> <strong>Insertion</strong>
<strong>The</strong> obvious reason for the high level <strong>of</strong> failure with technology insertion programmes is
that there are a considerable number <strong>of</strong> risks inherent to projects <strong>of</strong> this type. Some <strong>of</strong>
these risks may be well understood and actively managed, but evidence suggests that
others may not be discovered even once the project has ended and the technology
introduced into service. <strong>The</strong> following section summarises the literature on risks that
have been found to impact upon the success <strong>of</strong> technology insertion projects.
5.1 Background
<strong>The</strong> complexity <strong>of</strong> technology programmes is well understood in disciplines such as
S<strong>of</strong>tware Engineering (SE), IS and IT. <strong>The</strong> significance <strong>of</strong> risks for IS/IT project success
was recognised by McFarlan [29] as early as 1974. He highlighted three serious
deficiencies in practice that were responsible for IS/IT project failures:
1. Failure to assess individual project risk;
2. Failure to consider the aggregate risk <strong>of</strong> the portfolio <strong>of</strong> projects;
3. Lack <strong>of</strong> recognition that different projects require different managerial approaches.
<strong>The</strong> complexity <strong>of</strong> most computer programs and programming languages is one <strong>of</strong> the
unsolved problems in s<strong>of</strong>tware engineering. <strong>The</strong> current applications are complex to the
extent that when programmers leave, companies fail, if there is no one else who
understands what the previous programmer has done. Researchers have conducted a great
deal <strong>of</strong> research in an attempt to develop metrics, which adequately capture and reduce
the complexity <strong>of</strong> the s<strong>of</strong>tware.
<strong>The</strong> early risk assessment models concentrated on technical factors as the causes for
failure. Earl [30] proposed a rigid cost-benefit framework for assessing on economic,
technical and operational criteria the viability, risks and opportunities represented by the
IT investment. Corder [31] identified high risk factors <strong>of</strong> project size, project definition,
user commitment and stability, project time, and the number <strong>of</strong> systems interfaces. Cash,
McFarlan and McKenney [32] refined this and suggested that the three most important
dimensions that influence the risk <strong>of</strong> a development project were project size, experience
with the technology and project structure. In general, it was found that the smaller, more
experienced and more highly structured the project then the less risk is associated with it.
One <strong>of</strong> the first models to identify organisational factors was by Parker, Benson and
Trainor [33], which highlighted the need to consider how well equipped the organisation
is to implement the project in terms <strong>of</strong> personnel, skills and experience.
5.2 Technical Risks
One area that deals with the introduction <strong>of</strong> technologies is the discipline <strong>of</strong> Information
Systems (a branch <strong>of</strong> s<strong>of</strong>tware engineering). Available literature suggests that failure is a
ubiquitous feature <strong>of</strong> both the UK and international experience <strong>of</strong> IS engineering. In fact
failure is such a problem in IS programmes that it has become a major focus <strong>of</strong> attention
for both researchers and practitioners alike. Of course technology insertion risks are not
limited solely to IT and IS projects. <strong>The</strong>re is a plethora <strong>of</strong> evidence to suggest that TI
12

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
risks occur just as frequently in material, component and physical technology insertion
projects as in IT and IS projects. <strong>The</strong> use <strong>of</strong> IT and IS examples here is simply due to the
extent with which these two domains have analysed TI failures through the use <strong>of</strong>
detailed case studies. <strong>The</strong>se case studies are particularly valuable as they seek to
understand the causes <strong>of</strong> problems on IT and IS projects as they take a qualitative as well
as a quantitative approach to understanding the underlying issues.
5.2.1 Project Size
Willcocks and Griffiths [34] reviewed the main research studies on risk and concluded
that prominent projects commonly experience overruns, are <strong>of</strong>ten over budget, do not
perform in the way expected, or are cancelled prior to their completion after the
expenditure <strong>of</strong> considerable sums <strong>of</strong> money [35], [36], [37], [38]. Small projects run less
risk and are <strong>of</strong>ten more tolerant <strong>of</strong> process lapses, while large projects run exponentially
larger risk and need ever increasing management and control <strong>of</strong> risk. A 1994 survey <strong>of</strong>
200 IT projects found that for projects over £660,000, 90 per cent were over budget, 98
per cent had changed specification, 60 per cent were over time, and 20 per cent were
inappropriate [39].
5.2.1.1 Pressure on Managers and Implementers
Miller [40] reported that to implement IT innovation, top administrators are expected to
take the risk <strong>of</strong> failure or delay <strong>of</strong> IT adoption.
An assessment <strong>of</strong> five IT projects in the Danish public sector found that the customers as
well as the developers have had unrealistic expectations about how easily the systems
could be installed and put into operation [41]. Research has also suggested that IT project
managers may not be equipped to lead projects that are expected to transform an
organisation, and are poorly equipped to set technology priorities that affect more than
one part <strong>of</strong> the organisation [42]. Another problem is the over optimism and unrealistic
mindset <strong>of</strong> upper management which can come from insufficient understanding <strong>of</strong> the
technology and a lack <strong>of</strong> awareness <strong>of</strong> the magnitude <strong>of</strong> the impact that the new
technology will have upon the organisation.
5.2.1.2 Escalation
<strong>Technology</strong> projects have a tendency to escalate, with decision makers electing to
commit additional resources to a project where financial and organisational outcomes as
well as common sense indicate that the project should be cancelled or redirected. While
escalation <strong>of</strong> commitment is a general phenomenon that can occur with any type <strong>of</strong>
project, the literature [43], [44], [45], [46], [47] suggests that IT projects may be
particularly susceptible to this problem. <strong>The</strong> intangible nature <strong>of</strong> s<strong>of</strong>tware makes it
difficult to obtain accurate estimates <strong>of</strong> the proportion <strong>of</strong> work completed which may
promote escalation <strong>of</strong> commitment by giving a false perception that successful
completion <strong>of</strong> the project is near. To add to the difficulty <strong>of</strong> measuring progress, IT
projects are dynamic and tend to have volatile requirements [44], [46], which cause
project scope to change frequently. Almost certainly, projects subject to such volatility
are especially difficult to manage and control. For these reasons, it is not surprising that
escalation occurs with high frequency in IT projects. Project failure in the IT area is a
costly problem and troubled projects that seem to take on a life <strong>of</strong> their own are not
13

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
uncommon [48]. Prior research has shown that managers can easily become locked into a
cycle <strong>of</strong> escalating commitment to a failing course <strong>of</strong> action [49].
5.2.2 Maturity <strong>of</strong> <strong>Technology</strong>
In 1999, the United States (US) General Accounting Office (GAO) produced an
influential report that examined the differences in technology transition between the
Department <strong>of</strong> Defense (DoD) and private industry [50]. It concluded that the DoD takes
greater risks and attempts to transition emerging technologies at lesser degrees <strong>of</strong>
maturity than does private industry. <strong>The</strong> GAO concluded that use <strong>of</strong> immature technology
increased overall program risk and recommended that the DoD adopt the use <strong>of</strong> National
Aeronautics and Space Administration’s (NASA) <strong>Technology</strong> Readiness Levels as a
means <strong>of</strong> assessing technology maturity prior to transition. <strong>Technology</strong> Readiness Levels
(TRLs) have been used within the NASA as part <strong>of</strong> an overall risk assessment process
(since the late 1980s). By the early 1990s, they were routinely used to support technology
maturity assessments and comparisons <strong>of</strong> maturity between different hardware
technologies.
In the UK TRLs and broader System Readiness Levels (SRLs) have been adopted by the
(then) DPA to assess the maturity <strong>of</strong> evolving technologies (materials, components,
devices, etc.) prior to incorporating that technology into a system or subsystem and
continue to be used within technology procurement projects to this day. <strong>The</strong> primary
purpose <strong>of</strong> using TRLs is to help management in making decisions concerning the
development and transitioning <strong>of</strong> technology. Advantages include:
o Provides a common understanding <strong>of</strong> technology status
o Risk management
o Useful to making decisions concerning technology funding
o Useful to support decisions concerning transition <strong>of</strong> technology
Although many project teams have found TRLs to be useful, several sources cite the
difficulties in applying TRLs to assess the readiness <strong>of</strong> s<strong>of</strong>tware-based technologies and
products.
Some <strong>of</strong> the characteristics <strong>of</strong> TRLs that limit their utility in assessing COTS s<strong>of</strong>tware
products include the fact that readiness does not necessarily fit with appropriateness or
technology maturity. Smith [51] points out that ‘readiness’ and ‘maturity’—though
frequently used interchangeably—are not the same thing. A mature product may possess
a greater or lesser degree <strong>of</strong> readiness for use in a particular system context than one <strong>of</strong>
lower maturity. Numerous factors must be considered, including the relevance <strong>of</strong> the
products’ operational environment (usage patterns, timeliness/throughput requirements,
etc.) to the system at hand, product-system architectural mismatch, as well as other
factors.
5.2.3 Human Centred Design
<strong>The</strong> National Audit Office (NAO) report by the Controller and Auditor General [52],
addresses the reasons why many equipments accepted into service by the MoD do not
fully meet the operational requirements. This report provides examples <strong>of</strong> projects where
14

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
operational difficulties can be attributed to Human Factors (HF) issues [53]. It also
supplies financial cost information associated with these difficulties and makes specific
references to HF.
<strong>The</strong> executive summary <strong>of</strong> the NAO report states that a survey <strong>of</strong> equipments accepted
into service by the MoD over a 5-year period showed that 40 per cent fully met the
operational requirements. In half <strong>of</strong> the remaining 60 per cent <strong>of</strong> cases where the
equipments did not fully meet the operational requirements, the MoD made concessions
which removed the obligation for the equipment suppliers to make good the shortfall,
either because <strong>of</strong> pressure from end users or because the shortfalls were not considered to
affect operational capability. In the other half <strong>of</strong> cases where the equipments did not fully
meet the operational requirements the suppliers accepted responsibility for making good
the shortfall.
<strong>The</strong> executive summary <strong>of</strong> the NAO report states that:
‘<strong>The</strong>se acceptance strategies and project risk assessments tended to emphasise the
importance <strong>of</strong> integration risks, while underplaying potential problems with reliability,
environmental testing, the quality <strong>of</strong> individual components and human factors’.
Bruseberg [53] found that besides aiming at performance improvements, and usable
products, HF activities can have much wider effects in the organisation including:
• reducing major costs;
• avoiding redesign costs;
• improving mission success rate;
• reducing accident levels;
• improving efficiency <strong>of</strong> development and implementation processes;
• mitigating operational risks.
When it comes to new technologies being introduced into military organisations, it is
clear that the human factor must be considered carefully. One example <strong>of</strong> where
technology can confuse if it is not carefully designed is illustrated by the example <strong>of</strong> a
very senior naval <strong>of</strong>ficer [54] experiencing difficulties during the Iraq campaign. <strong>The</strong>
<strong>of</strong>ficer stated that owing to the much wider use <strong>of</strong> e-mail services in lieu <strong>of</strong> the timehonoured
‘signal’, he was sometimes ‘a little uncertain’ whether messages constituted
‘advice’ or a ‘direct order to act’.
5.2.4 Interoperability <strong>of</strong> Components
Introducing modifications and updates to major military platforms, such as aircraft, is a
time-consuming and costly activity. Some authors have suggested that each modification
or replacement is treated independently, sometimes with disregard for overall system
capability. <strong>The</strong>y argue that equipment may be introduced that will not be integrated
effectively with other present and future elements, and will almost certainly not match the
needs <strong>of</strong> networking. Marshall and Underhill [55] found that, belatedly, the total aircraft
is being recognised as a system.
15

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
<strong>The</strong>re are also considerable risks to IT system success due to issues <strong>of</strong> compatibility and
interoperability. For example, in 1996 the United States (US) DoD created ‘Joint Vision
2010’, which listed high-tech capabilities it hoped to acquire. In 2000 a modified version
called Joint Vision 2020 was released [56]. This version relies on IT and several authors
have criticised such an approach, arguing that it has led to ‘infrastructures that are
incompatible and unreliable’.
Gentry [57] provides examples <strong>of</strong> some <strong>of</strong> the interoperability challenges that the DoD
faces:
‘<strong>The</strong> sheer size and organisational complexity <strong>of</strong> the DoD IT infrastructure make
achievement and maintenance <strong>of</strong> interoperability and security a daunting task— even
without the complication <strong>of</strong> attacks. DoD has over 10,000 computer systems… DoD has
some 1.5 million individual computers, most <strong>of</strong> which are networked; to keep abreast <strong>of</strong>
changing technology, about a third are replaced each year. S<strong>of</strong>tware is upgraded
regularly. Hundreds <strong>of</strong> organisations procure and operate the equipment.’
Despite unified acquisition procedures, US Defense efforts to achieve department-wide
system interoperability, and ‘homilies about the virtues <strong>of</strong> jointness’, the services and US
Defense agencies ‘refuse to obey the spirit and letter <strong>of</strong> sometimes long-standing
policies’ and continue to buy systems for their use alone. In late 2000, some $36 billion
in planned acquisition was reportedly not interoperable. During operations in the former
Yugoslavia, US forces used some 30 IS systems that, according to the Defense Science
Board, were only ‘integrated into a loose federation <strong>of</strong> collection capabilities.’
According to Gentry, programme managers <strong>of</strong> key systems are not responsible for
assuring interoperability with other systems and would be ‘out <strong>of</strong> their lanes’ if they tried.
While nominally the organisational chief information <strong>of</strong>ficers and agency heads have
such responsibilities, in practice the acquisition <strong>of</strong> single systems occurs largely
independently. This sometimes leads to what some DoD IT pr<strong>of</strong>essionals call ‘drive-by
fieldings’—surprise delivery <strong>of</strong> IT for which users are neither technically nor financially
prepared.
For the same reasons, programme managers and their agencies do not systematically
address enterprise-wide consequences <strong>of</strong> their systems, including basic ones like the
impact <strong>of</strong> their systems on the IT infrastructure and the impact on DoD’s limited stock <strong>of</strong>
IT pr<strong>of</strong>essionals. Often they ‘do not care whether there is adequate bandwidth to operate
their systems; that is somebody else’s problem’. <strong>The</strong>se interoperability issues are argued
to have negative consequences for US defense capability with:
• infrastructures that are vulnerable to attack;
• high IT infrastructure failure rates (even when not under attack);
• failure to apply military technology successfully during wartime;
• high risks <strong>of</strong> overwhelming ground commanders due to sheer volume <strong>of</strong> data.
Gentry [57] concludes that the DoD’s ‘electronic system-based force structure is
expensive, fragile, and vulnerable.’
16

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
5.2.4.1 Maintainability <strong>of</strong> System
When bringing in a new technology into a system, factors such as system maintenance
need to be considered. With the focus on Network Enabled Capabilities (NEC) some
authors have raised concerns over the reliance upon IT support pr<strong>of</strong>essionals in keeping
military systems running.
It has been suggested that the DoD’s recruitment and retention <strong>of</strong> IT pr<strong>of</strong>essionals has
been ‘so ineffective in recent years that a working group chaired by the Under Secretary
<strong>of</strong> Defense for Personnel and Readiness is addressing the issue. <strong>The</strong>re is no solution in
sight. DoD cannot now conduct normal IT operations well, let alone surge to support a
national emergency or deal with attacks’ [57].
Current government procurements <strong>of</strong> large, complex systems emphasise the use <strong>of</strong> COTS
products because they <strong>of</strong>fer four main advantages. First, COTS products demonstrate
feasibility prior to start <strong>of</strong> development. <strong>The</strong>y virtually eliminate development risk. In
many cases, it is possible to ‘try before buying’ and select the ‘best’ implementation from
among competing products. COTS products also <strong>of</strong>fer ‘higher quality implementations <strong>of</strong>
functions than do custom-developed systems’. This is because COTS products are usually
tested across multiple systems, and there is usually considerable effort on the part <strong>of</strong> the
developers to maintain and improve the quality <strong>of</strong> the product. By eliminating the need to
develop components, COTS products enable shorter development schedules for new
systems. <strong>The</strong> final advantage is that the development costs for COTS products are
‘amortized over large numbers <strong>of</strong> systems, thereby reducing the cost to each user’ [58].
This cost reduction continues throughout the life cycle as bug fixes and enhancements are
incorporated into upgrades.
It is ‘important to recognise’ that the above advantages ‘do not occur automatically’
when COTS products are used. Each system’s architecture and product selections need to
be carefully tailored in order to avoid the numerous potential pitfalls that can dilute the
above advantages.
Others have identified the need to pay careful attention to component obsolescence when
selecting electronic equipment.
<strong>The</strong> speed <strong>of</strong> development <strong>of</strong> electronics hardware has risen dramatically and those
involved in the maintenance <strong>of</strong> military platforms face the problem that by the time the
platform is complete, many <strong>of</strong> its sub systems and components would be ‘at best obsolete,
and probably no longer in production. By the time the US Navy fielded the Aegis weapon
system in 1978, over half <strong>of</strong> the hardware used was no longer available.’
Several authors have highlighted the risks associated with the use <strong>of</strong> COTS electronics,
which are not designed to cope with the demanding environments and rugged treatment
that military systems encounter.
As Langley [58] describes, the classic mil-spec procurement method meant that the
government financed the development <strong>of</strong> a product that had to be rugged enough for
military use. ‘<strong>The</strong> government owned and controlled the design, but paid a high penalty
in time and money.’
While problems <strong>of</strong> COTS ruggedisation are not insurmountable, they do need to be
considered before technologies are inserted into military systems. Designs that use COTS
products must consider the harsh environments in which these commercial devices may
17

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
be used. Many COTS suppliers <strong>of</strong>fer hardware built to meet greater levels <strong>of</strong> heat, cold or
vibration. Such ruggedised hardware may use commercial parts that are able to withstand
extremes <strong>of</strong> heat, cold and humidity, or can be qualified to do so.
Evidence suggests that the selection <strong>of</strong> technologies for insertion into systems and
platforms must consider wider issues than initial cost, performance and capability [59].
Technologies must be able to cope with demanding military contexts, and there must be
systems in place to provide technical support to ensure capabilities are maintained. If
sufficient technical support cannot be provided or is prohibitive, decisions are needed
over whether complex technologies, such as those used in NEC, are appropriate or
sufficiently reliable.
<strong>The</strong> use <strong>of</strong> widely accepted commercial standards for interfaces and architectures helped
support the acceptance <strong>of</strong> COTS as effective military equipment solutions. <strong>The</strong>se
standards allowed the creation <strong>of</strong> open systems interfaces and architectures that were
public domain rather than company propriety. Open standards, free to use by hardware
manufacturers encourage competition between s<strong>of</strong>tware and hardware vendors. This
encourages the development <strong>of</strong> other products, using the same architecture, making it
relatively easy to switch from one vendor to another.
One disadvantage <strong>of</strong> using COTS equipment is the rapid pace <strong>of</strong> product development.
New s<strong>of</strong>tware and hardware can be introduced at intervals ranging from a few years to
less than a year. Procurers have to consider that whilst COTS products may <strong>of</strong>fer better
performance, they may not be rugged enough for use in military applications. Mature
COTS items that have been tested and used by the commercial market are more likely to
be suitable for military designs, but consequently may be more prone to obsolescence
issues by the time the system is fielded. When a new version <strong>of</strong> a COTS product is
released this can also be problematic as it may mean that the older product, which was
procured, will not be supported by the vendor for much longer. Military systems are
typically required to have a service life <strong>of</strong> 20 years or more and by using COTS
equipment additional risks to reliability and supportability emerge. <strong>The</strong> system designer
must be prepared to face COTS obsolescence problems during development, production
and service life.
Given the slow timescales <strong>of</strong> military development programmes, it is possible that
components or s<strong>of</strong>tware chosen early in development may no longer be available, or no
longer supported, when the system reaches production. <strong>The</strong>re are a number <strong>of</strong> approaches
to solving the component obsolescence problem, including making a one <strong>of</strong>f ‘lifetime’
buy <strong>of</strong> all the hardware which will be needed for the predicted production run and for
spares. This ‘is a workable system but may not fit with procurement funding. It
effectively locks the customer to a given level <strong>of</strong> performance, and denies them the cost
savings that might come from declining prices’. [58] Another approach is for the supplier
to enter into an agreement with component and subsystem vendors asking for prior notice
<strong>of</strong> the forthcoming obsolescence <strong>of</strong> any key component. This allows the end user a last
chance to buy components, but it relies heavily upon individual vendors fulfilling their
agreement to warn <strong>of</strong> upcoming item obsolescence.
Langley argues that product availability is an important consideration when looking at
COTS products. Vendors <strong>of</strong>ten sell products that are coming to market in the near future
and new products can <strong>of</strong>ten arrive late and sometimes may not arrive at all. Similarly
Langley warns that customers have very little control over vendor schedules, product
capabilities, or product availability and should plan accordingly.
18

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
Langley lists other key considerations for COTS products, including the need to get
hands on experience <strong>of</strong> product functionality, ‘while vendor literature and demonstrations
contribute to an understanding <strong>of</strong> product functionality, there is no substitute for
extensive hands on experience.’ Langley argues that this is the only way to determine
whether generalised capabilities provided by a COTS product meet the specific customer
requirements. This consideration is perhaps given extra credence by the problems that the
National Programme for IT (NPfIT) encountered when trying to procure and insert COTS
technology throughout the NHS [60]. In an effort to digitise patient records in the NHS, a
COTS s<strong>of</strong>tware product, developed for use in the US healthcare system was procured.
<strong>The</strong> project is a public private partnership (PPP) between the Trust and a US based
s<strong>of</strong>tware house (USCo) contracted to supply, configure and support their customisable<strong>of</strong>f-the-shelf
healthcare information system in cooperation with an in-hospital project
team.
Although there were many other issues in the NPfIT, one <strong>of</strong> the major stumbling blocks
was the fact that the COTS product had been designed around the US model <strong>of</strong> health
care. Key differences in healthcare provision between the two countries caused
significant challenges to implementing the new s<strong>of</strong>tware. One <strong>of</strong> the major differences
between the NHS and US healthcare is that US healthcare services are not ‘free at the
point <strong>of</strong> use’, so the COTS user interface had sections dedicated to payment and
insurance details, which were not relevant in the UK. Trying to fit a US (insurance and
payment) oriented system to the UK is– ‘like fitting a square peg in a round hole.’ [60].
Some authors feel that the use <strong>of</strong> COTS can be considered a security risk. Richardson
[59] quotes Dr Stephen D Bryan, former Director <strong>of</strong> the DoD’s Defense <strong>Technology</strong>
Security Administration, who warned that ‘if we are selling COTS on an unrestrained
basis globally, then any strategic advantage we gain… will be immediately <strong>of</strong>fset. COTS
will become a security risk instead <strong>of</strong> a security benefit’.
5.3 Organisational Risks
<strong>The</strong>re is now an increasing body <strong>of</strong> evidence to suggest that organisational issues play a
significant role in system failures. Jeffcot and Johnson [61] found that a lack <strong>of</strong>
consideration <strong>of</strong> organisational issues in systems development can cause project failure.
Organisational risks can be defined as the ‘non-technical aspects <strong>of</strong> systems development,
which might have an impact on the ultimate success or failure <strong>of</strong> a project’ [62], [13].
Considering these aspects as risks seems sensible, given the evidence, Ewusi-Mensah and
Przasnyski [19], for example, found that organisational factors such as ‘senior
management involvement’ and the ‘degree <strong>of</strong> end-user participation in the project
development’ were the ‘most widespread and dominant factors contributing to project
failure’.
Although the importance <strong>of</strong> organisational issues is well understood, especially in
disciplines such as socio-technical systems, research shows that systems development is
more <strong>of</strong>ten than not, technology led. Consequently organisational issues are not properly
addressed [6], [13], and too much responsibility rests with IT pr<strong>of</strong>essionals. <strong>The</strong>
implications <strong>of</strong> this are illustrated by Hornby [63] who argues that ‘Systems analysts do
not claim to have knowledge <strong>of</strong> organisational issues in IT systems, and there is no
evidence that they are encouraged or rewarded for considering such issues’. <strong>The</strong> literature
backs up the view <strong>of</strong> these authors that systems development methodologies typically
encourage the implementing <strong>of</strong> a system, and then simply hope that they will be able to
cope with the organisational implications later.
19

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
Mackenzie’s [64] belief that in order ‘To make computer systems safer, we need to
address not merely their technical aspects, but also the cognitive and organisational
aspects <strong>of</strong> their real-world application’ seems entirely appropriate.
5.3.1 Organisational Barriers to <strong>Technology</strong> Changes
<strong>Technology</strong> research suggests that there are organisational and technological factors that
limit the adoption and use <strong>of</strong> technologies. Indeed there is evidence that numerous
aspects, such as organisational size, resources, management support, and innovation
history influence the extent <strong>of</strong> technology adoption. Researchers have also been aware for
a long time that the introduction <strong>of</strong> new technology can <strong>of</strong>ten result in changes to
people’s roles within the organisation. Several authors have identified what they refer to
as ‘barriers’ to organisational change and hence technology adoption. <strong>The</strong> following
section will discuss what barriers may be encountered, where these ‘barriers’ appear and
what their consequences can be in terms <strong>of</strong> technology project success.
5.3.1.1 Acquisition Processes
Surprisingly perhaps, an organisation’s own acquisition processes can lead to risks to the
success <strong>of</strong> technology programmes. Military acquisition processes have been particularly
problematic when it comes to technology insertion projects. For example Marshall and
Underhill [55], whilst discussing the Tornado platform, state that ‘Many capability
requirements cannot be met . . . because the current processes for both sustainment
modifications and capability upgrades take too long and . . . cost too much’. <strong>The</strong>y argue
that the defence procurement system is ‘perceived as overly bureaucratic, inefficient and
lacking in pragmatism and flexibility’.
Similarly, Skorczewski [65], chairman <strong>of</strong> the Industrial Avionics Working Group, argues
that established arrangements for providing the Armed Services with weapons and
equipment are not designed to stimulate technology insertion. He goes on to state that
‘purchasing rules, procedures and culture are all actively discouraging. <strong>The</strong> defence
industry is particularly affected yet is expected to respond when required with the skills,
resources and innovation to match MoD’s needs’.
It would appear that the UK is not alone in this respect, with Gentry [57] suggesting that
the lack <strong>of</strong> centralised accounting <strong>of</strong> DoD equipment means that there is not sufficient
management <strong>of</strong> the DoD’s IT infrastructure.
5.3.1.2 <strong>Technology</strong> Adoption; Demographics, Politics and Fear <strong>of</strong> Change
<strong>Technology</strong> adoption, technology acceptance and the diffusion <strong>of</strong> innovations are
domains <strong>of</strong> study in their own rights. Work in these areas has revealed that organisational
politics can play a role in technology adoption. A number <strong>of</strong> authors have found that
managers and personnel will actively resist technology changes if they feel they would
lose power and decision making authority. <strong>The</strong>re are a number <strong>of</strong> ways that this
resistance might occur, for example Edmondson et al [66] found that those with power
would influence others’ views by conveying their thoughts about the implications <strong>of</strong> a
new technology.
Other researchers have similarly found that one <strong>of</strong> the key ways in which organisational
politics may affect the ‘post-implementation success <strong>of</strong> a technology’ is through user
20

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
resistance. Keen [67] describes a number <strong>of</strong> ‘counter-implementation strategies’ that
users may take to impede the development or implementation <strong>of</strong> the system:
‘Lay low’. If you do not want a system to succeed, then the more you keep out <strong>of</strong> the way
and do not give help and encouragement, the more likelihood there is <strong>of</strong> failure.
‘Rely on inertia’. If you can be too busy when asked, then the implementation process
may come to a halt.
‘Keep the project complex, hard to coordinate and vaguely defined’. If the goals are
ambiguous or too ambitious, there is every chance <strong>of</strong> failure as energy is dissipated in
many different directions.
‘Minimise the implementers’ legitimacy and influence’. If the designers are kept as
outsiders, other users will probably not allow them to work effectively.
‘Exploit their lack <strong>of</strong> inside knowledge’. <strong>The</strong> design team probably know very little about
the detailed nature <strong>of</strong> the work and if they are denied this knowledge, the system will
probably prove to be inadequate when it is implemented.
Tolbert and Zucker [68] report that innovation <strong>of</strong> information technology is more likely
to succeed when the ‘political environment to which an organisation belongs has norms
favouring the change’. <strong>The</strong>y found that attempts at improving IT facilities <strong>of</strong> government
organisations depended on whether support from administrative authorities, top
administrators, local and central government was available for IT managers. Similarly
Kim and Bretschneider [69] report that support from administrative authorities plays a
significant role in whether the innovation efforts are frustrated or completed.
People’s personal reasons for ignoring, resisting or even sabotaging new technology have
long been considered in organisational research. Indeed, resistance to change is a widely
recognised phenomenon [70]. Some researchers have found that staff may under-use, or
even sabotage new technology if they feel they may be adversely affected by it.
At the individual level, the most important attitudinal factors leading to resistance are fear
<strong>of</strong> change [71], [72], [73] and computer-related anxiety [73], [74]. Fear <strong>of</strong> change is
expressed through ‘concern about safety, security, or self-esteem’. It is manifested
primarily through worrying about loss <strong>of</strong> skill or possible replacement by more efficient
equipment. <strong>The</strong> second attitudinal factor, anxiety, is a natural feeling <strong>of</strong> uneasiness when
exploring or facing unfamiliar situations. This feeling can be ‘intensified when
confronted with new technology’.
Attributing the rejection <strong>of</strong> innovations only to anxiety and fear <strong>of</strong> change, however, is an
oversimplified view <strong>of</strong> the process <strong>of</strong> technology transfer [75]. One person’s perception
<strong>of</strong> ‘user intransigence’ is another person’s ‘looking after your own interests’. Change
management is <strong>of</strong>ten used to try to overcome issues surrounding the ‘fear’ <strong>of</strong> change.
However, in some cases, especially when considering those impacted by changes, it may
be appropriate for people to resist changes. Users may be correct in resisting
technological changes if the technologies do not suit the tasks, procedures and culture <strong>of</strong>
the organisation or where the new technology will not bring benefits to themselves, or
support their needs.
Evidence suggests that where technologies provide a benefit or are useful, users react
positively to their implementation. Technologies such as email, for example, have proven
21

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
vastly successful in many organisations, such that managers are now in a position where
they have to discourage its use due to the heavy weight <strong>of</strong> email traffic. Carey [76] finds
a correlation between acceptance <strong>of</strong> change and variables such as previous use
(experience), education, and current usage <strong>of</strong> a new system. She also reports
commitment, exposure to change and preparation for change are important for successful
implementation <strong>of</strong> new technologies and systems. Eason mentions the mismatch to
organisational purposes and functions, as a possible reason for failure to accept a
particular technology [77].
5.3.1.2.1 Diffusion <strong>of</strong> Innovation <strong>The</strong>ory
Rogers’ [78] Diffusion <strong>of</strong> Innovations (DoI) theory was developed in an attempt to
describe the patterns <strong>of</strong> adoption <strong>of</strong> new technologies and to help predict whether a new
invention will be successful [79]. In this model, Rogers classifies individual adopters into
five categories regarding their innovativeness (i.e. their likelihood <strong>of</strong> adopting an
innovation):
1) Innovators; 2) Early Adopters; 3) Early Majority; 4) Late Majority; and 5)
Laggards.
<strong>The</strong> temporal distribution <strong>of</strong> adopters <strong>of</strong> innovation follows a bell-shaped curve, see
figure 1, below.
Figure 1: Rogers E.M. 'Diffusion <strong>of</strong> Innovations' (1983)
According to Rogers, the five adopter categories are characterised by three sets <strong>of</strong>
variables; socio economic status, personality variables and communication behaviour.
Based on meta-analysis <strong>of</strong> those factors in different innovation-diffusion settings, early
adopters <strong>of</strong> innovations were found to be, on average, younger individuals with higher
socio-economic status, higher level <strong>of</strong> intelligence and rationality, more open to change,
and more knowledgeable about innovation.
Moore and Benbasat [80], working in an Information Systems context, expanded on these
categories, generating eight factors (voluntariness, relative advantage, compatibility,
image, ease <strong>of</strong> use, result demonstrability, visibility, and trialability) that impact the
adoption <strong>of</strong> IT. Research has shown however that innovation adoption rates are impacted
by other phenomena. For instance, the adaptation <strong>of</strong> technology to individual needs can
change the nature <strong>of</strong> the innovation over time. In addition, a new innovation can impact
the adoption rate <strong>of</strong> an existing innovation. New innovations may mean that the adoption
<strong>of</strong> a previous innovation is not taken up by some or is abandoned by others.
22

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
5.3.1.2.2 <strong>Technology</strong>, Usefulness and Predicted Behaviour
<strong>The</strong> <strong>Technology</strong> Acceptance Model (TAM) describes how users come to accept and use a
technology. <strong>The</strong> model suggests that when users are presented with a new technology, a
number <strong>of</strong> factors influence their decisions about how and when they will use it.
TAM is an adaptation <strong>of</strong> Ajzen and Fishbein’s [81] theory <strong>of</strong> reasoned action (TRA),
which proposes that perceived usefulness and perceived ease <strong>of</strong> use determine an
individual's intention to use a system [82]. TRA is an important model from social
psychology that focuses on the drivers <strong>of</strong> consciously intended behaviours. According to
TRA, behavioural intention and attitude drives an individual’s performance. TAM
replaces many <strong>of</strong> the TRA attitude measures with the technology acceptance measures,
ease <strong>of</strong> use, and usefulness. See Figure 2 below.
Figure 2: Venkatesh et al’s <strong>Technology</strong> Acceptance Model (TAM)
Some researchers have found TAM to account for around 40% <strong>of</strong> user acceptance [83].
TAM2 extended the original model to explain perceived usefulness and usage intentions
in terms <strong>of</strong> social influence and cognitive instrumental processes. <strong>The</strong> extended model
was tested in both voluntary and mandatory settings. <strong>The</strong> results strongly supported
TAM2 and the authors were able to explain 60% <strong>of</strong> their adoption model using this
updated version <strong>of</strong> TAM [84].
TRA and TAM assume that when someone forms an intention to act, that they will be
free to act without limitation. In reality however, constraints such as limited ability, time,
environmental or organisational factors, and unconscious habits limit people’s freedom to
act [85]. Yet the existing research on TAM presents inconsistencies and <strong>of</strong>fers relatively
low explanatory powers. Researchers have started to question the generalisability <strong>of</strong>
TAM [83], [86], [87]. [88]. Moderating factors such as age, gender, experience,
characteristics <strong>of</strong> technology [87], among other factors [89], [90] have been shown to
account for the inconsistent relationships.
In an attempt to integrate the main competing user acceptance models, Venkatesh et al.
formulated the Unified <strong>The</strong>ory <strong>of</strong> Acceptance and Use <strong>of</strong> <strong>Technology</strong> (UTAUT). This
model was found to outperform each <strong>of</strong> the individual models [84].
An alternative diffusion <strong>of</strong> innovation theory, the ‘Unified <strong>The</strong>ory <strong>of</strong> Acceptance and Use
<strong>of</strong> <strong>Technology</strong>’ (UTAUT), aims to explain user intentions to use technology as well as
23

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
explain subsequent usage behaviour. <strong>The</strong> theory holds that four key constructs
(performance expectancy, effort expectancy, social influence, and facilitating conditions)
are direct determinants <strong>of</strong> usage intention and behaviour [82]. Gender, age, experience,
and ‘voluntariness’ <strong>of</strong> use are posited to mediate the impact <strong>of</strong> the four key constructs on
usage intention and behaviour. See Figure 3, below [84].
Figure 3: Unified <strong>The</strong>ory <strong>of</strong> Acceptance and Use <strong>of</strong> <strong>Technology</strong> (UTAUT).
Despite the explanatory appeal <strong>of</strong> these models, there are a number <strong>of</strong> limitations <strong>of</strong> DoI
<strong>The</strong>ory. Clarke [91] argues that ‘at its best (DoI) is a descriptive tool, (it is) less strong in
its explanatory power, and less useful still in predicting outcomes, and providing
guidance as to how to accelerate the rate <strong>of</strong> adoption’.
<strong>The</strong>re is also some doubt about the extent to which DoI theory can give rise to readily
refutable hypotheses. On top <strong>of</strong> this, diffusion <strong>of</strong> innovation theory has been criticised for
the fact that ‘many <strong>of</strong> its elements may be specific to the culture in which it was derived
(such as North America in the 60s)’ and that it is ‘less relevant in, for example, East
Asian and African countries’ [91].
5.3.1.3 Organisational Norms and Cultures
Organisational culture and climate are found to impact attitudes toward innovation
adoption both for technology [77] and mode <strong>of</strong> practice [92]. Organisational culture and
climate are related to organisational processes, service quality, client outcomes, and
worker attitudes, perceptions, and behaviours [93], [94], [95].
In the US, some authors [96], [97] have argued that the military has so far ‘failed to
match the rhetoric <strong>of</strong> transformation with action’. While each (US military service)
claims to embrace new ways <strong>of</strong> war, ‘none has yet demonstrated a sustained commitment
to fundamental change’. Nothing shows this more clearly than military acquisition
budgets. Service funding is still dominated by incremental improvements to traditional
systems; radically new technology, doctrine, and organisations have received smaller
resources. <strong>The</strong> authors argue that ‘none <strong>of</strong> this should be surprising. Large bureaucracies
such as the US armed forces are designed to minimise uncertainty, including that brought
24

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
on by large-scale change. And new is not always better’. <strong>The</strong> National Security Agency
(NSA) is an intriguing test case <strong>of</strong> the struggle between a reformer and bureaucratic
inertia. NSA director Hayden reportedly concluded that the root causes <strong>of</strong> NSA’s major
IT failure in early 2000 were managerial in nature; for example, NSA then had five
largely autonomous directorates and 68 e-mail systems at Fort Meade alone. Hayden
attacked leadership problems by bringing in new senior managers, reorganising, and
developing new technology in a program called Trailblazer. In July 2001, NSA finalised
its ‘Groundbreaker’ program <strong>of</strong> outsourcing IT by awarding a $2 billion, ten-year
contract to Computer Sciences Corporation. <strong>The</strong>se efforts led to internal resistance;
Hayden's adviser, James Adams, estimated that 25 per cent <strong>of</strong> NSA personnel supported
Hayden, another 25 per cent opposed him, whilst the rest were 'fencesitting'. Similarly,
then Secretary <strong>of</strong> Defense Donald Rumsfeld in mid-2001 encountered military opposition
to his review <strong>of</strong> national defense priorities and stated that, for the military, ‘change is
hard.’ Under the best <strong>of</strong> circumstances, even vigorous reform efforts will bear fruit
slowly.
Following an organisation’s decision to adopt a technology, users’ perceptions and
managers’ attitudes affect their willingness to use it, which affects implementation
success [98]. Successful implementation has been defined as the incorporation or routine
use <strong>of</strong> a technology on an ongoing basis in an organisation [99], [100]. Many studies
emphasise the need for organisations to adapt for a new technology to be used effectively
[100], [101], [102], [103], [104]. Leonard-Barton [98] described a need for mutual
adaptation by both organisations and technologies. For many technologies, new
knowledge must be transferred to enable use – not just technical knowledge but social
knowledge about who knows what [102], [105]. In addition, technology adoption occurs
in stages, presenting different hurdles to adoption over time [100]. Evidence from a range
<strong>of</strong> studies suggests that ensuring adoption <strong>of</strong> new technologies in organisations is not
straightforward.
25

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
6 <strong>Impact</strong>s <strong>of</strong> <strong>Technology</strong> on the Organisation
In addition to the risk <strong>of</strong> failure and <strong>of</strong> overrun budgets and timescales, the impact that
technology has upon organisations continues once the technology has been introduced.
This section seeks to summarise the various impacts that occur as well as the theoretical
concepts surrounding technology and the organisation. <strong>The</strong> impact <strong>of</strong> technology on
organisations is a heavily studied subject and includes the domains <strong>of</strong> organisational and
management science, change management, socio-technical systems, computer science,
human factors and many others besides.
An exercise in the book ‘Human Computer Interaction’ [106] asks the reader to ‘spend a
couple <strong>of</strong> minutes writing the advantages that you think computers have provided to
people and organisations . . . then write a second list <strong>of</strong> the various problems that
computerisation has brought’. <strong>The</strong> author then goes on to conclude ‘Both lists are
endless’. If this is the case for the impact <strong>of</strong> computers, it is certainly true <strong>of</strong> the impact
that technologies can have upon organisations.
In the following section the key impacts that technology has had upon organisations are
summarised. <strong>The</strong> list is by no means complete, as there are simply too many facets to
cover in any scoping study; however, the intention is to give an overview <strong>of</strong> some <strong>of</strong> the
key issues impacting on organisations.
6.1 Performance and Productivity
Despite the fact that many organisations invest in information technology in order to
reduce costs and to increase productivity, there is considerable debate in the literature
over whether such outcomes have been delivered. <strong>The</strong>re has been much discussion over
whether a ‘productivity paradox’ exists, especially regarding the amount <strong>of</strong> productivity
delivered by information technology (IT). This paradox has been used to describe the
evidence suggesting that despite heavy investment in IT for many years, the rate <strong>of</strong>
measured productivity growth has failed to increase, and may have even decreased. Lehr
and Lichtenberg [107] argue that ‘since productivity is defined as output per unit <strong>of</strong> input,
and computers are an input, we should start by asking under what conditions one would
expect growth in computer intensity to raise productivity’.
Evidence also suggests that it costs a lot <strong>of</strong> money to rapidly and successfully integrate
new s<strong>of</strong>tware systems. In 1999, Berger [108] found that Micros<strong>of</strong>t spent $16,000 per
annum for each <strong>of</strong> its workstations on maintenance and upgrading. Supporters <strong>of</strong> the
productivity paradox view claim that trying to transform the way work is done and
simultaneously save money is ‘usually a mistake’. In 1990, the famous economist Robert
Solow [109] stoked the debate further by saying that ‘We see computers everywhere
except in the productivity statistics’.
Despite this, there is evidence to support the counter argument that new technologies,
including computers and s<strong>of</strong>tware do improve productivity.
According to Strassman [110] ‘most businesses well endowed with IT lose about $5000
per year per workstation as a result <strong>of</strong> the need for futzing’. Futzing refers to ‘the time
users spend in a befuddled state while clearing up unexplained happenings and
overcoming the confusion and panic when computers produce enigmatic messages that
stop work’.
26

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
Home <strong>of</strong>fice workers and distributed team members lack peer support, so the amount <strong>of</strong>
time spent futzing increases substantially. Notably, the futz factor is <strong>of</strong>ten hidden from
those at the top. For example, at Xerox the most common encounters between senior
management and copiers is when new machines are presented for their inspection.
Inevitably in such encounters, people whose jobs relied on it going well surrounded the
managers. Consequently, as managers experimented, the correct finger was edged toward
the correct button at the correct time in a thousand barely perceptible ways. What
everyone took to be a successful encounter between an individual, instructions, and the
copier, was actually an encounter made successful by an informal and almost invisible
social network working hard to prevent the embarrassment <strong>of</strong> failure to all [111].
In recent years, a surge in the number <strong>of</strong> studies that examine the IT pay<strong>of</strong>f is a testimony
to the challenge <strong>of</strong> measuring whether a productivity paradox exists [112], [113], [114],
[115], [116], [117].
One <strong>of</strong> the counters to the productivity paradox argument is that we may fail to measure
productivity gains from computers because there is a substantial time lag before gains are
realised. David [118] argues that computers may require substantial changes in
complementary infrastructure (such as human and knowledge capital and global
communications infrastructure) before the gains can be realised.
6.1.1 Usability
<strong>The</strong> usability industry has emerged as a direct result <strong>of</strong> the fact that computer-based
products and systems are difficult to use. Usability consultant Jakob Nielsen and
computer science pr<strong>of</strong>essor Ben Shneiderman have developed a framework <strong>of</strong> system
acceptability [119], where usability is a part <strong>of</strong> ‘usefulness’ and is composed <strong>of</strong>:
• Learnability (e.g. intuitive navigation);
• Efficiency <strong>of</strong> use;
• Memorability;
• Few and noncatastrophic errors;
• User satisfaction.
Usability includes considerations such as:
• Who are the users, what do they know, and what can they learn?
• What do users want or need to do?
• What is the general background <strong>of</strong> the users?
• What is the context in which the user is working?
• What has to be left to the machine? What to the user?
• Can users easily accomplish their intended tasks?
• How much training do users need?
27

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
• What documentation or other supporting materials are available to help the user?
• What and how many errors do users make when interacting with the product?
• Can the user recover from errors? What do users have to do to recover from
errors? Does the product help users recover from errors?
• Are there provisions for meeting the special needs <strong>of</strong> users with disabilities?
(accessibility).
Usability techniques have been developed in order to answer these questions including
methods such as: user-focused requirements analysis, building user pr<strong>of</strong>iles, and usability
testing. With its origins in human factors, usability engineering has had considerable
success improving productivity in IT organisations. For instance, a major computer
company spent $20,700 on usability work to improve the sign-on procedure in a system
used by several thousand people. <strong>The</strong> resulting productivity improvement saved the
company $41,700 the first day the system was used. On a system used by over 100,000
people, for a usability outlay <strong>of</strong> $68,000, the same company recognised a benefit <strong>of</strong>
$6,800,000 within the first year <strong>of</strong> the system’s implementation. This is a cost-benefit
ratio <strong>of</strong> 1:100 [120].
6.2 Manpower Levels and Organisational Size
As we have seen, a key argument for the investment in technology is in order to achieve
increases in organisational productivity. Traditionally this has been achieved using
machines to replace the productivity <strong>of</strong> the workforce and to reduce labour costs. In
recent years much <strong>of</strong> the research has focused on the impact <strong>of</strong> information technologies
on organisation size.
<strong>The</strong>re is substantial evidence <strong>of</strong> a relationship between increased levels <strong>of</strong> IT usage and
smaller organisation size, suggesting that IT systems reduce the level <strong>of</strong> manpower
needed. Brynjolfsson et al [7] found that the overall relationship is robust to a variety <strong>of</strong>
specifications and at least four measures <strong>of</strong> firm size. However, they argue that ‘findings
should not be interpreted to apply to all industries and all time periods’. <strong>The</strong> decline in
firm size is greatest with a lag <strong>of</strong> one to two years following investments in IT,
suggesting that the impacts <strong>of</strong> the new technology are not fully felt immediately. This
finding may shed light on previous studies that found little or no impact <strong>of</strong> IT in the same
year that the investments were made.
Another possible explanation for why IT might lead to smaller firms is that IT might
allow firms to ‘outsource’ more <strong>of</strong> their activities. In other words, the use <strong>of</strong> IT might
lead firms to ‘buy’ rather than ‘make’ more <strong>of</strong> the components and services needed to
make their primary products [7]. <strong>The</strong> obvious conclusion is that in some cases technology
support services are sometimes outsourced to consultancies. It is perhaps self evident that
these consultancies can range in size from small operations to global companies such as
IBM or CSC.
A number <strong>of</strong> studies <strong>of</strong> the relationship between technology and employment find
evidence that IT may actually increase employment. Osterman [121], for example, found
that IT investment ultimately resulted in an increase in the number <strong>of</strong> clerks and
managers employed after a lag <strong>of</strong> several years. Similarly, Morrison and Berndt [122]
found that IT was on balance a complement, not a substitute for labour, especially white-
28

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
collar labour. Specifically, they conclude: ‘. . . rather than being aggregate labour-saving,
increases in IT tend to be labour-using’.
It seems that whilst there are many specific studies showing a loss <strong>of</strong> jobs, overall
assessments such as those made by the US Congress Office <strong>of</strong> <strong>Technology</strong> Assessment
[123] conclude that ‘technology was probably generating more jobs than it was
destroying’.
It would appear that predicting the impact <strong>of</strong> manufacturing technologies is much more
straightforward. As far back as the 1950s, Woodward [124] developed a measurement
scale on which firms were characterised according to the technical complexity <strong>of</strong> the
manufacturing process. In this instance, high technical complexity indicated that most <strong>of</strong>
the work was carried out by machines, whereas low technical complexity meant that
workers play a greater part in the process. Woodward found that the number <strong>of</strong>
management levels and manager to personnel level increased as technological complexity
increased. In non-manufacturing industries, where the output is less tangible, the
relationship between manpower levels and technology is much less predictable.
6.3 Distribution <strong>of</strong> Power and Control
It has long been assumed that IT will lead to flatter organisational structures and the
sharing <strong>of</strong> power. However, Zub<strong>of</strong>f [125] and Lohr [126] have found that the ‘paradise <strong>of</strong>
shared knowledge and a more egalitarian working environment is just not happening. If
power is not shared, it is because management does not want to share authority and
power’. <strong>The</strong>y each conclude that this is a result <strong>of</strong> organisational culture, rather than
technology.
Studies by Fukuyama and Shulsky [127] examined the relationship between
disintermediation (removing the middleman), flat organisations, and centralisation on
behalf <strong>of</strong> the US Army. <strong>The</strong>y found that the conventional argument that IT will lead to
flatter organisations is dependent on a single function <strong>of</strong> middle management: the
aggregation, filtering, and transmission <strong>of</strong> information. In other words, if all information
travels through middle management layers there will be centralisation <strong>of</strong> power.
Advances in IT suggest that flattening is desirable, since IT facilitates the automation <strong>of</strong>
much <strong>of</strong> this work. This research found that if the role <strong>of</strong> managers in an organisation is
primarily information processing, then information-processing equipment might replace
them, and organisations will become flatter. If, on the other hand, there is more to the
manager’s role than information processing, then linear predictions about flattening <strong>of</strong> the
organisational structure are too simple.
Again it would seem that there is evidence to support both centralisation and
decentralisation arguments. In fact, an international study <strong>of</strong> the effects on power
distribution [128] found every possible outcome across an array <strong>of</strong> case studies:
centralisation, decentralisation, lateral transfer across departments and no impact at all.
6.4 Job Enrichment vs Deskilling
One common theme <strong>of</strong> new technologies in organisations is that they aim to substitute
machine technology for human labour. Automatic Teller Machines (ATMs), for example,
29

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
have replaced thousands <strong>of</strong> human bank tellers and IBM has built a factory that can
produce laptops without the help <strong>of</strong> a single worker. Besides replacing production,
technologies have led to job simplification, where the variety and difficulty <strong>of</strong> tasks
carried out are reduced. This can lead to boredom and reduced job satisfaction. More
advanced technologies on the other hand tend to cause job enrichment, meaning that the
job provides greater responsibility, recognition and opportunities for growth and
development. <strong>The</strong> introduction <strong>of</strong> ATMs into banks took most <strong>of</strong> the routine tasks
(deposits and withdrawals) away from bank tellers and left them with the more complex
tasks that require higher level skills. <strong>The</strong>se technologies create a greater need for
employee training and education because workers need higher level skills and greater
competence to master their jobs.
Some research has shown that the introduction <strong>of</strong> computing technologies into almost all
workplaces has made the nature <strong>of</strong> work more complex [129]. <strong>The</strong> increasing complexity
<strong>of</strong> modern organisations and substantial knowledge needed to get the job done has led to
greater demand for a highly skilled workforce with distinct technological expertise and
knowledge [130].
Even when the aim <strong>of</strong> implementing IT is not to change the organisational structures and
processes it still has a knock on effect on the social system and culture. Social and
technical systems are interrelated and the implementation <strong>of</strong> IT involves the introduction
<strong>of</strong> a technical system into a social environment. <strong>The</strong>re are a number <strong>of</strong> issues that need to
be considered when implementing technology that will impact upon job design and
individual or group performance. Eason [62] identifies five categories:
1. Task Support Problems: <strong>The</strong> failure to look in detail at the work <strong>of</strong> each type <strong>of</strong>
user <strong>of</strong>ten means that the system delivers an inappropriate service to the end user.
2. Job Content Issues: Implementation requires changing the user’s job. <strong>The</strong>
distribution <strong>of</strong> work may be affected and the burden <strong>of</strong> work may be increased in
unexpected areas as a result <strong>of</strong> the new system. For example, as salespeople are
able to process more orders following the implementation <strong>of</strong> a new orders system,
this may put a greater burden on the distribution and delivery <strong>of</strong> the product.
3. Formalisation: Systems require more standardised and formal procedures across
all users in order to provide commonality.
4. Power and Influence: <strong>The</strong> availability and access to information will change
relationships and power influences.
5. Personnel Policies: <strong>The</strong>re may be a need to retrain or employ different staff to
operate the new system. Different skills may be required as the job content and
workload changes to meet the needs <strong>of</strong> the new system. This will also impact on
career pr<strong>of</strong>iles and future training requirements.
In the 1980s researchers tried to resolve the argument about whether new technology
deskilled or enriched existing jobs. Research by various authors [131], [5], [132], [133]
indicates that:
• new technology sometimes enriches jobs and sometimes simplifies them;
• both effects can occur within the same organisation;
30

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
• simplification is more common than enrichment, especially amongst shop floor
manufacturing jobs;
• jobs are usually changed in some respects by new technology. New skills, such as
abstract thinking, computer programming and understanding <strong>of</strong> the organisation’s
systems are <strong>of</strong>ten required.
6.5 Information Management and Policy
Within both UK local and central Government, the implementation <strong>of</strong> new technologies
raises new and different information management and policy challenges. It is also found
to increase public expectations with respect to information access and service delivery.
New information policy issues generated by IT use, also influence practices as new
legislation and regulations are developed that affect the way organisations collect, use
and disseminate information. As government is also a substantial market for the IT
industry, its requirements and uses <strong>of</strong> IT have an effect on industry development <strong>of</strong> new
technologies and applications.
6.6 <strong>Impact</strong> at the Individual and Group Level
It is increasingly accepted that good Human Factors is critical to ensure that system
performance is safe, effective, and efficient.
<strong>The</strong>re are a multitude <strong>of</strong> impacts that new military technologies have had on individual
and group capability. For example the introduction <strong>of</strong> night vision goggles has meant that
military missions can be carried out throughout both the day and night. <strong>The</strong> impact on
individuals and teams has led to improved performance <strong>of</strong> tasks at night. An associated
impact on individuals is that missions extending further into the night may lead to less
sleep and increased fatigue. At a group level this might require greater consideration <strong>of</strong>
sleep management issues.
<strong>The</strong> introduction <strong>of</strong> new missile technology on aircraft with the capability to target and
engage enemy aircraft beyond the pilot’s visual range has resulted in several human
consequences. <strong>The</strong> most obvious being that pilots find it harder to successfully identify
enemy aircraft. Identification and differentiation, already a difficult task, is made more
challenging when engaging beyond visual range. Unsurprisingly, technologies are being
developed to try to aid the pilot in this task.
A lack <strong>of</strong> focus on Human Factors issues in the design and integration <strong>of</strong> new
technologies can lead to impacts upon through-life costs and military capability [53]. For
example, the SA80 Rifle and Light Support Weapon was beset by a series <strong>of</strong> problems
over a period from 1985 to 1992. <strong>The</strong> estimated cost for the modifications to fix these
problems was £24 million [134]. A significant proportion <strong>of</strong> the problems was related to
operability/fitness for purpose, including:
• accidental discharge – the weapon could accidentally discharge if dropped on its
muzzle when the safety catch was on;
• trigger reassertion – sometimes the weapon’s trigger had to be manually flicked
back into position after firing;
31

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
• bipod deployment – the weapon’s bipod stand would sometimes accidentally fall
down from its stowed position;
• problems with bayonet – it was possible for the bayonet to fall <strong>of</strong>f if the weapon
had been left resting on the bayonet release catch; it was difficult to sharpen;
some bayonets broke <strong>of</strong>f at the tip; the wire cutters on the bayonet were
inadequate;
• magazine release catch – the weapon’s magazine could be accidentally released
by the release catch snagging on webbing/clothing etc;
• cleaning kit – the cleaning kit was found to be inadequate and had to be
completely replaced;
• reliability in sandy conditions – the weapon was prone to stoppages in sandy
conditions.
Relevant bodies <strong>of</strong> knowledge have emerged throughout the last century as a result <strong>of</strong> the
impact <strong>of</strong> technology upon the individual and upon the organisation. Domains such as
change management, socio-technical systems and human factors have developed
considerable bodies <strong>of</strong> knowledge as well as numerous methods and techniques to avoid
negative impacts upon the end users <strong>of</strong> technologies. <strong>The</strong>y have also sought to maximise
the performance <strong>of</strong> man and machine. It is beyond the scope <strong>of</strong> this document to detail all
relevant aspects <strong>of</strong> these domains, or to comment upon their success rate. Rather it is to
raise awareness that the challenges <strong>of</strong> technology insertion are not new and that valuable
work and research has already been carried out in this area, albeit under different names
and from a range <strong>of</strong> perspectives.
32

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
7 <strong>The</strong>ories <strong>of</strong> <strong>Technology</strong> <strong>Impact</strong>
<strong>The</strong> early theoretical work in this area assumed technology to be an objective, external
force that would have (relatively) deterministic impacts on organisational properties such
as structure. In contrast, a later group <strong>of</strong> researchers focused on the human aspect <strong>of</strong>
technology, seeing it more as a product <strong>of</strong> the choices and intentions <strong>of</strong> implementers.
Work on technology shifted to a ‘s<strong>of</strong>t’ determinism where technology is seen as an
external force with impacts moderated by human actors and organisational contexts.
Recently attention has begun to focus on a re-conceptualisation <strong>of</strong> the ‘impact’ <strong>of</strong>
technology in a way that attempts to integrate features <strong>of</strong> both perspectives. Such
integrationist models portray an impact as ‘a complex, interactive and ongoing process
not as a simple linear outcome’. <strong>Technology</strong> does not ‘impact’ on its social environment
or vice versa but, over time, each shapes the other [135].
<strong>The</strong>re are also alternative viewpoints emerging which focus their attention on explaining
the processes that occur in complex systems. In a move away from traditional
reductionist approaches, some theorists have framed technologies and organisations as
complex systems in order to explain and understand the causes <strong>of</strong> unexpected and
emergent behaviours.
7.1 <strong>The</strong>ories <strong>of</strong> <strong>Technology</strong> in Organisations
7.1.1 Optimists and Pessimists
<strong>The</strong> question <strong>of</strong> how technology impacts humans and organisations has been the subject
<strong>of</strong> considerable debate over the years and it is a subject that Eason tackled in an article
written for the journal ‘Behaviour and Information <strong>Technology</strong>’ [24]. On the one hand,
Eason states, there are optimists such as Englebart [136] who foresee exciting forms <strong>of</strong>
‘man computer symbiosis’ in which the power <strong>of</strong> the computer will augment human
capacities to process information. In the organisational setting this was seen as
empowering people and increasing job satisfaction as computers were used to enable
people to realise much more <strong>of</strong> their potential. By contrast pessimistic authors in the
1970s predicted ‘the collapse <strong>of</strong> work’ [137], a nightmare world in which computers
would be used to automate work and replace human labour, throwing large numbers <strong>of</strong>
secretaries, clerks and even managers and pr<strong>of</strong>essionals out <strong>of</strong> work. For those who
remained at work the ‘centralisation <strong>of</strong> power’ or ‘big brother’ hypothesis applied [138],
the computer would be used to monitor and control the work <strong>of</strong> all employees leading to
widespread dissatisfaction and alienation [139].
One <strong>of</strong> the major concerns has been the impact <strong>of</strong> technology upon jobs, and, since the
1980s there have been two prominent views [62] [140], as follows:
1. Technical System as Control - <strong>Technology</strong> is seen as deskilling, taking work from
people and reducing the remainder to tedious and repetitive work.
2. Technical System as Tool - This view sees technology as enriching, whereby the
routine and boring jobs are allocated to computers, which also provide tools to allow
people to be creative and handle information in diverse and powerful ways.
33

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
7.1.2 <strong>Technology</strong> Led <strong>The</strong>ories
One <strong>of</strong> the most influential technology led models, the ‘technological imperative’ model
examines the impact <strong>of</strong> a technology upon organisational dimensions such as structure,
size, performance, degree <strong>of</strong> centralisation as well as dimensions such as job satisfaction,
task complexity, skill levels and productivity. This model states that technology exerts an
independent, uni-directional and causal influence over humans and organisations similar
in nature to the laws <strong>of</strong> physical sciences [141]. Some technology led theories argue that
society itself is entirely determined by technology: ‘new technologies transform society
at every level, including institutions, social interaction and individuals. Technological
determinism is a school <strong>of</strong> thought believing that technology is the single most important
factor in determining the success <strong>of</strong> an organisation’ [142]. This approach has also been
known, in economics and elsewhere, as ‘technology push’. It asserts that investing in the
latest technology is the only way for an organisation to survive and notions such as
‘automate or liquidate’ and the fear <strong>of</strong> being ‘left behind’ are examples <strong>of</strong> this approach.
In this body <strong>of</strong> work, studies <strong>of</strong> technology and information technology examine the
impacts <strong>of</strong> technology on organisational dimensions such as structure, size, performance,
and centralisation/decentralisation, as well as individual level dimensions such as job
satisfaction, task complexity, skill levels, communication effectiveness, and productivity.
<strong>The</strong> premise is that the technology as well as the organisational and individual variables
can be measured and predicted [124], [143], [144], [145], [146], [147], [148], [149],
[150], [151], [152], [153]. It is worth noting that much <strong>of</strong> this research is now over 25
years old.
Such research treats technology as an independent influence on human behaviour or
organisational properties that exerts unidirectional influence over humans and
organisations, similar to those operating in nature [154]. While providing insight into the
<strong>of</strong>ten determining aspects <strong>of</strong> technology, this body <strong>of</strong> research largely ignores the action
<strong>of</strong> humans in developing, appropriating, and changing technology. Some <strong>of</strong> the more
recent research, [155], [156], allows for the influence <strong>of</strong> technology to be moderated by
contextual variables, proposing a contingency model <strong>of</strong> technology’s effects. <strong>The</strong>
technological imperative model is illustrated in Figure 4.
Figure 4: Technological Imperative Model (from Orlikowski 1991)
Technological determinism stands in opposition to the theory <strong>of</strong> the social construction <strong>of</strong>
technology, which suggests that both the path <strong>of</strong> innovation and the consequences <strong>of</strong>
technology for humans are strongly if not entirely shaped by society itself, through the
34

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
influence <strong>of</strong> culture, politics, economic arrangements, and the like. Technological
determinism has been largely discredited within academia, especially by science and
technology studies [156]. However, it remains the dominant view within most news
media and popular culture.
7.1.3 Strategic Choice
In contrast to technological determinism, there is a school <strong>of</strong> thought which argues that
technology does not determine human action, but that rather, human action shapes
technology [157], see Figure 5. This perspective suggests that technology is not an
external object, but a product <strong>of</strong> ongoing human action, design, and appropriation.
Research focuses on how a particular technology is physically constructed through the
social interactions and political choices <strong>of</strong> human actors. <strong>Technology</strong> is seen as a
dependent variable, contingent on other forces in the organisation, most notably powerful
human actors. This perspective does not accept that technology is given or immutable,
focusing attention instead on the manner in which technology is influenced by the context
and strategies <strong>of</strong> technology decision makers and users [158], [159], [160], [161], [162],
[163].
Figure 5: <strong>The</strong> Strategic Choice Model (from Orlikowski 1992)
Particularly relevant to the strategic choice approach are socio-technical studies, which
are carried out in the belief that outcomes such as job satisfaction and productivity <strong>of</strong>
workers can be manipulated by jointly ‘optimising’ the social and technical factors <strong>of</strong>
jobs [160], [164]. A similar premise runs through the socio-technical research in
information technology [165], [166]. While usefully demonstrating how the meaning <strong>of</strong>
technologies arise and are sustained, this body <strong>of</strong> research tends to downplay the material
and structural aspects <strong>of</strong> interaction with technology.
7.1.4 Integrationist Models
Various attempts to develop intermediate approaches that recognise the contribution <strong>of</strong>
both technological and social factors [167], [168] have been made in response to the
perceived deficiencies <strong>of</strong> extreme technological and social determinist positions. <strong>The</strong>se
35

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
include socio-technical systems [166], social shaping [169] and social construction <strong>of</strong>
technology [157].
A number <strong>of</strong> researchers and authors have found that both technology and the decisions
made in organisations bring about change. In large scale study <strong>of</strong> the introduction <strong>of</strong>
computers into different industries, Buchanan and Boddy [131] found that both
management goals and technological factors caused changes to the organisation.
Similarly, the impact on jobs will vary within any organisation. Some will be deskilled
and others enriched. <strong>The</strong>y found that different conditions will produce different changes
and interactions, making it very difficult to predict the outcome <strong>of</strong> introducing new
systems or organisational structures.
Eason [170] found that technology is very flexible and does not have deterministic effects
on organisations. <strong>The</strong> technology is best regarded as a contributory or facilitating factor
in the organisational outcomes that have been found. Three other sets <strong>of</strong> factors also
contribute:
Types <strong>of</strong> technology and application. <strong>The</strong> use <strong>of</strong> different types <strong>of</strong> technology may make
a difference. Mainframe and dumb terminals might lead to centralised control whilst the
use <strong>of</strong> networked personal computers might lead to decentralisation. Management control
applications should lead to tighter control whereas computer aided design, decision
support systems, etc . . . should ‘augment human intellect’. . .
<strong>The</strong> goals <strong>of</strong> the user organisation. A major determinant <strong>of</strong> impact is the set <strong>of</strong> goals to
which the user organisation aspires when it invests in the technology. If the intention is to
use the technology to replace jobs, that may be the outcome. If the intention is to create a
monitoring system to control employee behaviour, the same technology might lead to this
outcome. As the early socio-technical systems theorists put it, there is ‘organisational
choice’ [164]; the same technology can be used to produce different organisational
outcomes.
<strong>The</strong> response <strong>of</strong> the user community. Even the intentions <strong>of</strong> those investing in the
technology are, however, not sufficient to predict the outcomes. Many outcomes are
unplanned and unintentional. High failure rates and low utilisation levels are certainly not
planned and most staff reduction levels are not actually achieved. <strong>The</strong> new technical
system has to engage with the complex world <strong>of</strong> tasks, procedures and culture within the
organisation; it has to be part <strong>of</strong> a working socio-technical system. <strong>The</strong> many
stakeholders at the receiving end <strong>of</strong> the new system will be active in responding to the
technical system to avoid negative consequences for themselves and where possible to
achieve benefits. This is most <strong>of</strong>ten perceived as resistance to change as stakeholders
defend against that which is not in their best interests. However, the response to the
technology may be positive as stakeholders find interesting ways to exploit the new
functionality. Many companies, for example, are currently groaning under the unplanned
weight <strong>of</strong> the e-mail traffic <strong>of</strong> their employees and are looking for ways <strong>of</strong> restricting it.
<strong>The</strong>re is still considerable debate in the literature regarding the type and extent <strong>of</strong>
influence that humans and technologies exert on organisational systems [171]. This issue
has been described as the ‘problem <strong>of</strong> agency’ and has been subject to considerable
debate as to whether humans and technologies should both be considered ‘agents’. This is
described by Orlikowski and Iacano [171] as one <strong>of</strong> the ‘key unresolved issues for our
field’.
36

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
Attewell [172] argues that ‘It is important to understand that because <strong>of</strong> the last three
decades <strong>of</strong> research and the importance <strong>of</strong> context, many distinguished scholars <strong>of</strong>
technology avoid the term ‘technology impact’. Using this term in framing the question is
viewed by some as indicating an ignorance <strong>of</strong> the body <strong>of</strong> scholarship in technology
studies. For them, the term ‘impact’ connotes a kind <strong>of</strong> technological determinism that is
very dated and widely discredited. As a result, the more recent models <strong>of</strong> technology are
descriptive <strong>of</strong> the various forces and triggers involved in organisational change and are
not normally intended as accurate tools for predicting specific technological impacts or
organisational outcomes.
7.1.5 Chaos <strong>The</strong>ory and Complexity <strong>The</strong>ory
Complexity theory, which has developed from Chaos theory, looks at how very simple
things can generate very complex outcomes that could not be predicted by just looking at
the parts by themselves. Chaos is sometimes viewed as extremely complicated
information, rather than as an absence <strong>of</strong> order. A key difference between the two
theories is that chaos theory remains deterministic, i.e. with perfect knowledge <strong>of</strong> the
initial conditions and <strong>of</strong> the context <strong>of</strong> an action, the course <strong>of</strong> this action can be
predicted. Complexity theory is non-deterministic, and gives no way whatsoever to
predict the future. Indeed it was developed to help researchers from a range <strong>of</strong> disciplines
to develop theories for situations too complex to be explained by earlier principles [174].
Scott [175] even suggests that complexity theory has brought ‘new vitality to many areas
<strong>of</strong> science where a more typical reductionist strategy has fallen short.’
Complexity theorists purport that we cannot predict what a complex system will evolve
into. <strong>The</strong>y argue that ‘all life from the smallest cell to the largest animals are complex
systems’ and have developed the term ‘emergence’ to describe the way complex systems
and patterns, such as those that form a hurricane, arise out <strong>of</strong> a multiplicity <strong>of</strong> relatively
simple interactions. Examples <strong>of</strong> emergence include ‘intelligence in the field <strong>of</strong> AI, or
agents in distributed artificial intelligence’, emergence is central to the physics <strong>of</strong>
complex systems and yet very controversial.
According to many complexity theorists, socio-cognitive systems, including humans,
groups and organisations, are all complex by their nature. <strong>The</strong> study <strong>of</strong> socio-cognitive
complexity, a relatively new domain in systemics, is beginning to look at complexity and
emergence within organisations. Complexity theory has been used extensively in the field
<strong>of</strong> strategic management and organisational studies, sometimes called 'complexity
strategy' or 'complex adaptive organisation'. Broadly speaking, complexity theory is used
in these domains to understand how organizations or firms adapt to their environments.
<strong>The</strong> theory treats organisations and firms as collections <strong>of</strong> strategies and structures. When
the organisation shares the properties <strong>of</strong> other complex adaptive systems - <strong>of</strong>ten defined
as consisting <strong>of</strong> a small number <strong>of</strong> relatively simple and partially connected structures -
they are more likely to adapt to their environment and, thus, survive.
One <strong>of</strong> the major contributions to early complexity theory is the distinction between the
human capacity to predict the behaviour <strong>of</strong> simple systems and its capacity to predict the
behaviour <strong>of</strong> complex systems through modelling [176]. Hayek believed that economics
and the sciences <strong>of</strong> complex phenomena in general, ‘including biology, psychology, and
so on, could not be modeled in the same manner as sciences that deal with essentially
simple phenomena like physics’. Hayek notably explains that complex phenomena,
through modeling, can only allow pattern predictions, compared with the precise
predictions that can be made out <strong>of</strong> non-complex phenomena [176].
37

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
Complexity theory may prove to be a useful approach to studying technology insertion
and the impacts that this can have upon organisations. If researchers feel they have
reached the point where it is impossible to predict organisational outcomes, due to the
complexity <strong>of</strong> the system, this approach may well be a valuable one. As complexity
theorists believe organisms, such as humans, and large organisations to be ‘complex
systems’ then it may be sensible for us to admit that, whilst we accept that there are many
factors that influence outcomes, it is impossible to predict with any accuracy the actual
final outcome. This <strong>of</strong> course does not mean that we should throw the baby out with the
bath water, so to speak. Decades <strong>of</strong> research tell us that some approaches work better than
others. Simply saying ‘it’s a complex system, let’s stand back and let it sort itself out’
would clearly be foolish as we know a great deal about the effective implementation <strong>of</strong>
technologies and how to minimise unpleasant consequences. What we do not know with
certainty is how all the actors in an organisation will interact, nor are we able to predict
the emergence <strong>of</strong> all organisational behaviours. Perhaps, in the future, complexity theory
will support more realistic ambitions by encouraging researchers to follow Hayeck’s view
and explore pattern predictions in complex systems rather than continuing to look for
precise predictions.
38

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
8 <strong>The</strong> Introduction <strong>of</strong> <strong>Technology</strong>
8.1 <strong>The</strong> Reality <strong>of</strong> <strong>Technology</strong> <strong>Insertion</strong> in Organisations
Organisational power politics play a crucial role in determining whether and how new
technology is introduced. In his research in engineering companies, Burnes [177] found
that the main motive for introducing new technologies was the belief that new technology
was ‘the future’, whilst very little thought was given to why this should be the case, or
whether technology was appropriate.
Langley and Taux [129] found that the adoption <strong>of</strong> new technology in small companies
seemed to involve three interrelated processes:
1. <strong>The</strong> strategic commitment process; an ‘incubation period’ during which
managers’ commitment to new technology fluctuates according to changes in
information, context and other decisions.
2. <strong>The</strong> technology choice process; this involves defining the technological needs and
priorities through investigation and analysis.
3. <strong>The</strong> financial justification process; which involves the preparation <strong>of</strong> formal
proposals for the choice <strong>of</strong> new technology, emphasising financial results and
market potential.
Blackler and Brown [178] identified three ways in which technology is introduced;
1. <strong>The</strong> muddle through approach. This approach results from management teams
that have no long-term goals and a very limited understanding <strong>of</strong> technology and
organisational design. Blackler and Brown discovered that the ‘muddle through
approach’ is surprisingly common.
2. <strong>The</strong> task and technology approach. This route is <strong>of</strong>ten perceived as good
management practice as it focuses on control, careful planning, cost-benefit
analysis and evaluation <strong>of</strong> new technology. However, it also takes a limited view
<strong>of</strong> the scope <strong>of</strong> the technology, with wider issues such as organisational impact
not being addressed.
3. <strong>The</strong> organisation and end user approach. Typically produces less predictable
solutions than the task and technology orientation. Underlying this approach is a
positive view <strong>of</strong> people and their value as well as a determination to only select
technologies that will meet wider organisational needs. <strong>The</strong> introduction <strong>of</strong> new
technology is embraced as an excellent opportunity to review and change existing
organisational structures and processes.
<strong>The</strong> organisation and end user approach involves considerable participation in system
design by potential users <strong>of</strong> new technology. This approach follows from the sociotechnical
systems theory and focuses on the idea that employees can contribute at all
stages <strong>of</strong> the process <strong>of</strong> bringing in new technologies. Processes may include
specification <strong>of</strong> system objectives, criteria to evaluate the new technology, pilot schemes,
evaluation studies and user support. Implementing this process is difficult to achieve for a
number <strong>of</strong> reasons. Although employees can provide valuable insight on what will and
39

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
will not work, technical experts may feel threatened by such an approach. Often experts
have a very well defined product to sell, which can only be modified very slightly to fit
the circumstances. A further difficulty is that technical experts can struggle in explaining
new technology in language that non-specialists can understand. This reduces the
opportunity for non-specialists to influence systems implementation projects. Blackler et
al [178] found that it is difficult to successfully implement a participative approach to
new technology and this is the case with the organisation and end user approach, in which
end user participation plays a key role.
8.1.1 Method <strong>of</strong> Implementation
<strong>The</strong> method <strong>of</strong> implementation will affect how users and organisations adapt to the
introduction <strong>of</strong> new technologies. Five key strategies for the implementation <strong>of</strong> IT can be
identified and are illustrated in Figure 6.
Revolution
Revolution
Evolution
Evolution
1. Big Bang
1. Big Bang
2. Parallel Running
2. Parallel Running
3. Phased Introduction
3. Phased Introduction
4. Trials and Dissemination
4. Trials and Dissemination
5. Incremental Evolution
5. Incremental Evolution
Figure 6: Implementation Planning (Source: Eason, 1988)
User Adaptation
User Adaptation
Difficult
Difficult
Easy
Easy
When a new system needs to be implemented, there are five different ways to adopt this
new system: the big bang approach, phased introduction, parallel running, trials and
dissemination or incremental evolution. <strong>The</strong>se are methods for delivering the application
or system to the end user; in themselves they do not constitute a strategy but a method or
structure for installing the system. <strong>The</strong> method chosen will depend upon the
circumstances <strong>of</strong> the organisation (time constraints, resources available), the environment
into which the system is being implemented (level <strong>of</strong> potential resistance, skills or
knowledge level) and the system design (technical capabilities). Stakeholder
considerations need to be taken into account as well as the requirements <strong>of</strong> the business
and costs. Selecting an implementation plan involves prioritising the goals to be achieved
and matching a strategy against these [62].
With the big bang adoption, the switch between using the old system and using the new
system happens at one single date, with an instant changeover <strong>of</strong> the system. Everybody
begins using the new system at the same time and the old system is discarded. <strong>The</strong> big
bang adoption type is riskier than other approaches as there are fewer learning
opportunities incorporated in the approach. Considerable preparation is needed to get this
to work smoothly [62]. This approach may be necessary where no previous IT system
40

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
existed or the business process does not allow any other method to be used concurrently.
Examples <strong>of</strong> such systems include the introduction <strong>of</strong> e-mail or ticketing systems. <strong>The</strong>
London Stock Exchange used a big bang approach in 1987 when it computerised trading.
Potential problems <strong>of</strong> the big bang approach may include complete system failure,
inappropriate modules and/or institutional shock.
Parallel running involves introducing the new system whilst continuing the old system in
tandem with it. This reduces the risk <strong>of</strong> the new system failing as the old one is available
as a backup. However, this approach doubles work requirements and therefore requires
greater resources and the added cost that this entails. All users can get used to the new
system, and fall back on their old system if necessary.
Phased adoption means that the introduction <strong>of</strong> new technology occurs in several phases.
In a phased approach aspects <strong>of</strong> the new system are introduced over a period <strong>of</strong> time.
Whilst this phasing in may be done on a geographical basis or user group basis, it is more
commonly done on a functional basis. <strong>The</strong> system is rolled out over a period <strong>of</strong> time,
building and developing functionality with each phase <strong>of</strong> the roll out. This is a common
approach with bespoke systems. Whilst it enables users to concentrate on specific
elements one at a time, problems may occur where functionality is linked, in new
networks, or where there is a mismatch <strong>of</strong> the old and new system. If minimising risk to
operation <strong>of</strong> the organisation is key, then either a parallel running or a phased
introduction approach may be appropriate.
<strong>The</strong> trials and dissemination approach is used where there are several different sites for
implementation. <strong>The</strong> system is implemented one site at a time as a series <strong>of</strong>
implementations. Sites may be selected for ease <strong>of</strong> implementation to ensure that the
correct approach is used and in order to iron out problems. However, sometimes different
sites can have different problems requiring a different approach. Another risk is that the
initial site selected to pilot the system may not be reflective <strong>of</strong> the other sites.
<strong>The</strong> final approach, incremental evolution, is a mixture <strong>of</strong> strategies and <strong>of</strong>ten involves
parts <strong>of</strong> a system being tested, whilst continuing to make changes. This approach is user
driven and is <strong>of</strong>ten beneficial for discretionary users, but can result in reduced
momentum. This approach can also be difficult to plan.
8.2 Risk Assessment in <strong>Technology</strong> Programmes
In Section 5 some <strong>of</strong> the key risks to technology insertion were outlined, however there is
evidence from case study research and from the literature [179], [180], [181] <strong>of</strong> an undermanagement
<strong>of</strong> risk in technology projects. Pike and Ho [181] found that there is a
general lack <strong>of</strong> managerial use <strong>of</strong> risk analysis in capital budgeting decisions to the
detriment <strong>of</strong> IT projects. Research studies have shown that the result <strong>of</strong> this is that over
20 per cent <strong>of</strong> IS expenditure is wasted and between 30 and 40 per cent <strong>of</strong> IS projects
realise no net benefits, however measured [183].
41

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
8.2.1 Risk Assessment Models
Risk assessment is the process involved in estimating the degree <strong>of</strong> risk associated with a
given project, usually at the feasibility stage <strong>of</strong> development [184], [185]. This approach
is designed to reduce risk; however, Lyyntinen and Hirsheim [186] found that a narrow
focus on risk techniques can blind participants to the organisational context, where many
projects encounter their most serious difficulties.
<strong>The</strong>re have been a number <strong>of</strong> risk assessment models developed specifically to analyse
risk in technology projects. Although most outline important characteristics <strong>of</strong> risky
technology projects, not all <strong>of</strong> them reflect the importance <strong>of</strong> organisational issues and
the overall environment and context within which the technology needs to function.
Environment and context are two important concepts, which need to be thoroughly
analysed and accommodated in design, if a technology is to function successfully within
an organisation.
<strong>The</strong> under-management <strong>of</strong> risks in IS projects is very surprising considering the size <strong>of</strong> IT
expenditure, and when the history <strong>of</strong> disappointed expectations is considered.
8.2.1.1 Willcocks and Margett's Risk Assessment Model
<strong>The</strong> basis <strong>of</strong> Willcocks and Margett's Risk Assessment Model came from work by
Pettigrew et al. [187], [188] on organisational change. Applying this premise, Willcocks
and Margetts developed six conceptual, interplaying categories that can be brought into
analysing the development, introduction and use <strong>of</strong> information systems:
History: Prior organisational development, e.g. relevant IS experience and organisational
history and most importantly, previous IS success or failure.
Internal Context: <strong>The</strong> characteristics <strong>of</strong> the organisation itself, e.g. strategy, structure,
reward system, management, human resources and industrial relations arrangements, IS
infrastructure and management.
External Context: <strong>The</strong> ‘givens’ that an organisation and its members need to respond to
and accommodate, e.g. the economy, political and governmental policy, markets,
competition and, in the public sector, department or local government guidelines,
procedures and funding arrangements.
Content: <strong>The</strong> changes involved in and substance <strong>of</strong> a project, e.g. size and complexity <strong>of</strong>
project, technical uncertainty, whether radical or incremental in impact.
Processes: How things are done and the issues perceived, e.g. project management,
project team experience, staffing stability, user commitment.
Risk Outcomes: Planned or unanticipated, desirable or otherwise, e.g. cost, time,
technical performance, operational efficiency, user acceptance.
How these categories link together is illustrated in Figure 7:
42

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
Figure 7 Interplaying Factors in the Model for Risk Assessment
8.2.1.2 Probabilistic Risk Assessment:
Willcocks and Margett’s [189] model, with its focus on environment and context, is
different to other established risk assessment methods such as Probabilistic Risk
Assessment (PRA). PRA is an integration <strong>of</strong> several techniques that aim to quantify and
assess the potential for failure and to help find ways to reduce risk. However, although
effective at revealing technologically centred failures, most PRA techniques do not
provide any explicit means <strong>of</strong> representing organisational or human factors failures [58].
PRA was designed for use in industrial and engineering environments, and some argue
that it is an inappropriate tool for identifying risk in more complex technology
programmes. Some authors have argued that established forms <strong>of</strong> risk assessment
generally neglect the organisational considerations involved in the implementation and
running <strong>of</strong> systems.
Leveson [190] reveals that policy and principles <strong>of</strong> management, organisational structure,
training factors, and the safety <strong>of</strong> the engineering process employed are just some <strong>of</strong> the
organisational factors that PRA neglects. She believes that PRA has ‘an inability to
represent these particular (organisational) aspects <strong>of</strong> the system or to evaluate them in the
analysis.’ <strong>The</strong>refore, to conclude, PRA techniques are too technologically centred to be
<strong>of</strong> use to assess risks in a military technology insertion programme. Willcocks and
Margett’s model has been described as being much more suitable for possible use in
complex technology insertion projects [58].
8.2.2 Knowledge Management
Knowledge Management (KM) comprises a range <strong>of</strong> practices used by organisations to
identify, create, represent, and distribute knowledge for reuse, awareness and
learning [201]. People starting a new project for an organisation can access KM resources
to learn best practices and lessons learned from previous projects or access relevant
information during the project. Managers can use KM resources to look for advice on
issues that they come across, or in order to access information after project completion
for advice on after-project actions and review activities. Knowledge management
practitioners <strong>of</strong>ten provide systems, repositories, and processes to encourage and
formalise these knowledge sharing activities.
43

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
<strong>The</strong> claims <strong>of</strong> KM supporters are that it allows organisations to be flexible and respond
more quickly to changing market conditions, and that it supports innovative, decision
making and productivity [202], [203].
One approach to managing and reducing technology project risks is to use KM techniques
to raise awareness <strong>of</strong> risks and problems and to stop people from repeating the mistakes
that others have made.
8.2.2.1 Performance, Challenges and Criticisms <strong>of</strong> KM
Evidence suggests that Knowledge Management systems rely upon employees’ ability
and willingness to transfer knowledge. Indeed, Szulanski [191] found that knowledge
transfer within a firm is inhibited by a number <strong>of</strong> factors and that the accessibility <strong>of</strong>
knowledge about best practices within a firm ‘depends upon the nature <strong>of</strong> that
knowledge, from where (or whom) it comes, who gets it, and the organisational context
within which any transfer occurs’ [191]. Knowledge Management has also been criticised
by those [192] who believe there are ‘serious issues surrounding employee power,
managerial control and the structure <strong>of</strong> knowledge work’.
<strong>The</strong> argument goes that with knowledge repositories, management will have more power
over employees. <strong>The</strong>se are consolidated into three propositions that the authors suggest
need more study:
P1. Use <strong>of</strong> knowledge repositories leads to reductions in employee uniqueness,
which, in turn, increases their substitutability and reduces their power.
P2. Use <strong>of</strong> knowledge repositories leads to reductions in the analytical skill
required in a job, which reduces the power position <strong>of</strong> the user.
P3. Management choice <strong>of</strong> control method moderates the impact that knowledge
repositories have on employee power.
Critics, such as Gray [193], argue that ‘the degree to which researchers and managers
understand the effects <strong>of</strong> knowledge repositories on the distribution <strong>of</strong> power will most
certainly influence the future success <strong>of</strong> such systems’.
A second problem for Knowledge Management is that it seeks to manage knowledge, ‘a
very slippery concept’ with many different variations and definitions. <strong>The</strong> nature <strong>of</strong>
knowledge, and what it means to know something, are epistemological questions that
have perplexed philosophers for centuries and according to ‘no resolution looms on the
horizon’. In fact, the field <strong>of</strong> knowledge management is itself controversial as there are
problems in the distinction between 'knowledge' and 'information'.
Wilson [194] argues that ‘Knowledge Management’ is an inappropriate term to describe
the field, as knowledge involves comprehension and understanding, something that
cannot be easily transferred from one person to another. <strong>The</strong> only way to transfer what
we wish to express is through language, communication and information. Wilson states
that ‘such messages do not ‘carry’ knowledge, they constitute information’. Hence the
term ‘Information Management’ is increasingly being used in preference to ‘Knowledge
Management’, although for consistency the latter will continue to be used in this
discussion.
44

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
Furthermore, KM relies heavily upon something called Case Based Reasoning (CBR), as
data is too scarce for statistical relevance. CBR, simply put, is the process <strong>of</strong> solving new
problems based on the solutions <strong>of</strong> similar past problems. <strong>The</strong> use <strong>of</strong> CBR as a method <strong>of</strong>
KM has been criticised as it is an approach that accepts anecdotal evidence as its main
operating principle. Without statistically relevant data for backing and implicit
generalisation, there is no guarantee that any generalisation is correct.
KM has also been criticised by those who debate the usefulness <strong>of</strong> experience [192]. <strong>The</strong>
argument goes that ‘much <strong>of</strong> our experience is <strong>of</strong> no value for future work’ because:
• Changing conditions makes history invalid for future predictions. Sometimes the
experience itself changes the conditions so much that the experience cannot be
mapped on to other cases.
• It is <strong>of</strong>ten impossible to compare what actually happened with what would have
happened if the actual actions were not taken. Establishing cause and effect is,
therefore, very difficult in one <strong>of</strong> a kind development projects.
• Experience is context dependent and much <strong>of</strong> the context is hard to describe.
Perhaps the biggest challenges to KM is that techniques rely upon the generation <strong>of</strong>
perceived risks or the interpretation <strong>of</strong> captured knowledge from colleagues and their
work on previous projects. This sort <strong>of</strong> knowledge capture is open to reporting
inaccuracies due to a number <strong>of</strong> cognitive biases which may, unintentionally, influence
people’s reported experiences. <strong>The</strong>se biases <strong>of</strong>ten occur subconsciously as a result <strong>of</strong>
normal human cognitive processes. To provide some background, a cognitive bias is any
<strong>of</strong> a wide range <strong>of</strong> observer effects identified in cognitive science and social psychology
including basic statistical, social attribution, and memory errors that are common to all
human beings. Biases drastically skew the reliability <strong>of</strong> anecdotal and legal evidence.
<strong>The</strong>re exist a number <strong>of</strong> biases, which have been consistently recorded in the literature.
Some <strong>of</strong> which will be described in the following paragraph. This is not an exhaustive
list, rather it includes those biases which are likely to influence reports <strong>of</strong> people’s own
performance on technology projects. Biases perhaps are inevitable in a context where
people are evaluated based on their own performance, and where they may be asked to
justify errors that might influence their career progression.
• Choice-supportive bias – the tendency to remember one's choices as better than
they actually were.
• Confirmation bias – the tendency to search for or interpret information in a way
that confirms one's preconceptions.
• Focusing effect – a prediction bias occurring when people place too much
importance on one aspect <strong>of</strong> an event; causes error in accurately predicting the
utility <strong>of</strong> a future outcome. This bias can cause error in accurately predicting the
utility <strong>of</strong> a future outcome.
• Anchoring or focalism – a cognitive bias that describes the common human
tendency to rely too heavily, or ‘anchor’, on one trait or piece <strong>of</strong> information
when making decisions. During normal decision making, individuals anchor, or
overly rely, on specific information or a specific value and then adjust to that
45

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
value to account for other elements <strong>of</strong> the circumstance. Usually once the anchor
is set, there is a bias toward that value. This is particularly likely to occur in KM
where a practitioner has considered others’ experience and is basing their
decisions and actions based on this ‘anchor’.
• Attentional bias – neglect <strong>of</strong> relevant data when making judgments <strong>of</strong> a
correlation or association.
• Neglect <strong>of</strong> prior base rates effect – the tendency to fail to incorporate prior
known probabilities which are pertinent to the decision at hand.
• Selective perception – different people perceive the same events differently.
Studies show that what we perceive is heavily influenced by what we expect and
want to find.
• Hindsight bias: When people know the actual outcome <strong>of</strong> a process, they tend to
regard that outcome as having been fairly predictable all along – or at least more
predictable than they would have judged before knowing the outcome. Experience
from s<strong>of</strong>tware projects collected after their completion can be strongly impacted
by hindsight bias (‘I-knew-it-from-the-beginning...’). Unfortunately, it is not
enough to inform people about hindsight bias and encouraging them to avoid it
[192].
Another challenge for Knowledge Management is that few studies have been done to
establish whether there is an empirical relationship between Knowledge and Business
performance [195]. Perhaps one <strong>of</strong> the reasons for this is that KM metrics have been
notoriously difficult to develop due to the intangible nature <strong>of</strong> the ‘knowledge resource’
[196]. As we have seen, ‘knowledge’ is difficult to define and has multiple interpretations
that make it difficult to value and measure. As a result <strong>of</strong> the complexity <strong>of</strong> assessing
organisational initiatives in general, research [197] and practice [198] on the assessment
<strong>of</strong> KM initiatives and knowledge management systems (KMS) is not well developed.
8.2.2.2 Learning From Experience
Learning From Experience (LFE), one approach to managing knowledge, has been
implemented within the MoD procurement and project management processes. LFE as
described on the MoDs Acquisition Management Site (AMS) [199] ‘identifies lessons
learned from past projects and how you can contribute to future learning’. LFE is a key
element <strong>of</strong> Knowledge Management (KM), concentrating on the knowledge sharing
aspect.
<strong>The</strong> AMS goes on to say that ‘Learning From Experience (LFE) is one <strong>of</strong> the Project
Delivery Assurance Criteria within the Project Review and Assurance (PR & A) Process,
seen as a key driver across all disciplines’. This LFE Guide has been ‘written to help
projects meet their business targets by applying LFE tools and techniques. It is an
effective Key Risk Management Tool, which helps with identifying potential risks and
issues.’
According to the MoD’s ‘Procurement Development Group Learning From Experience
Information and Process Guide’, there are a great number <strong>of</strong> Benefits from using LFE:
• It will develop individual staff by improving skills and knowledge.
46

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
• It is an effective Key Risk Management Tool, which helps with identifying
potential risks and issues.
• It will reduce the likelihood <strong>of</strong> making the same mistakes again.
• Individuals who contribute will be seen as ‘team players’.
• It will enable a project team to achieve its objectives.
• It will reduce time to perform tasks.
• It will reduce resources required to perform tasks.
• It will reduce stress in staff.
• It will improve project Planning and Scheduling through life.
• It will ensure projects are delivered on time, to cost and performance.
• It will provide measurable metrics <strong>of</strong> savings made to individual teams and to the
organisation as a whole.
An adage that the AMS uses to describe LFE goes: ‘You must learn from the mistakes <strong>of</strong>
others. You can’t possibly live long enough to make them all yourself’. <strong>The</strong> LFE pages
on the MoD AMS goes on to say that ‘Knowledge is a key, if not THE key, factor in
much <strong>of</strong> what we do. ‘Knowing what we know’ is vital if resources are to be used
efficiently and waste minimised. Effective knowledge management is not easy - it
demands support at all levels and permeates throughout the entire organisation.’
Whilst no one would argue that sharing knowledge and learning from others’ experiences
and mistakes can be no bad thing, research has found that knowledge management, and
consequently LFE, face some significant challenges.
As we have seen, the LFE program within the MoD’s AMS makes bold claims about
reducing stress, improving project performance, reducing time on tasks and managing
and mitigating risks. Significantly LFE says that it will ‘provide measurable metrics <strong>of</strong>
savings made to individual teams and to the organisation as a whole.’ This author would
be interested to know whether such metrics and performance data currently exist and
whether they suggests that LFE has been found to be effective so far. As we have seen
measuring the effectiveness <strong>of</strong> KM programmes is notoriously difficult, given the elusive
nature <strong>of</strong> knowledge. Evaluating the effectiveness <strong>of</strong> LFE as practiced within the MoD
does not fall within the scope <strong>of</strong> this review and so no data on its use, or its actual or
perceived value were identified. However, it is probably fair to say that although LFE has
potential benefits, it is unlikely to mitigate all possible technology project risks and
organisational impacts.
Regardless <strong>of</strong> the debates surrounding knowledge management, it is clearly no bad thing
for colleagues to try to impart knowledge to one another and to pass on pertinent
information. However, there are many challenges in this field and the capturing and
communication <strong>of</strong> expert knowledge and experiences is not something that is easily
achieved. Similarly, there are certain human characteristics that present serious
challenges to effective use <strong>of</strong> existing risk assessment and knowledge management
practices.
47

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
9 <strong>Technology</strong> <strong>Insertion</strong> Case Studies
<strong>The</strong>re are numerous examples <strong>of</strong> case studies where technologies have failed or not
achieved the desired effects once implemented. <strong>The</strong>re is not nearly as much detailed
examination <strong>of</strong> the processes that lead to technology failure. One area in which there is
more detailed consideration <strong>of</strong> failure cases is in the area <strong>of</strong> IS. <strong>The</strong> study <strong>of</strong> IS failure is
particularly useful, as many <strong>of</strong> the systems in question are funded with public money and
the result <strong>of</strong> investigations into the cause <strong>of</strong> failure are available in the public domain. In
other domains, such as the military, detailed information is less freely available and has
been predominantly gathered from subject matter experts. Where conclusions or findings
are described these have either been highlighted by the relevant case study, author or by
the subject matter expert (SME) describing the case.
9.1 Military
9.1.1 Eur<strong>of</strong>ighter Typhoon
When originally selecting a component for the Multi-function Head Down Display
(MHDD) for the Eur<strong>of</strong>ighter Typhoon, a number <strong>of</strong> component options were compared
including Cathode Ray Tube (CRT) and Active Matrix Liquid Crystal Display (AMLCD)
solutions. During comparisons, technical issues were considered including technology
maturity and performance <strong>of</strong> equipment including visibility factors such as luminance and
contrast ratio [204]. AMLCD components were considered but in the end, the technology
was considered too immature. High on the list <strong>of</strong> reasons were the difficulties LCD
experienced when operating over an extended temperature range, and a concern over the
long-term stability <strong>of</strong> the LCD cell. Other perceived problems at that stage in its
development were limited viewing angle, poor dynamic response and the low reliability
<strong>of</strong> fluorescent backlights. One <strong>of</strong> the biggest CRT component options was chosen,
measuring 6’ by 6’, which turned out to be non-industry standard dimensions. <strong>The</strong>
component in question seemed like a good choice at the time, but soon caused
difficulties. It was only manufactured by one company and the Typhoon developers were
tied to one supplier. On top <strong>of</strong> this the component in question was subject to complex US
International Traffic in Arms Regulations (ITAR) restrictions, which meant that the US
DoD were in a position where they could restrict the way in which the component and the
system was used. <strong>The</strong>se ITAR restrictions meant that the US could theoretically block the
export <strong>of</strong> Typhoon aircraft to countries not possessing an ITAR agreement. In the end a
decision was taken to replace the CRT displays with LCD components. <strong>The</strong> upgrade was
expensive, time consuming and could have been avoided if issues surrounding
international technology transfer, export agreements and supplier issues had been
considered at an earlier stage <strong>of</strong> development. In the end, the upgrade to LCD was
successful; a component was chosen that did not have ITAR restrictions and came with a
wider supplier base. Price competitiveness, supply and choice was protected and the
ITAR restrictions were no longer an issue. <strong>The</strong> technical issues with LCD were
overcome.
9.1.1.1 Findings
• Technical Issues such as product performance and maturity were given a high
priority.
48

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
• Export control issues and supply line issues were not anticipated or well understood.
• <strong>The</strong> need to design a flexible system allowing for component upgrades was not
considered sufficiently.
• <strong>The</strong>re was an insufficient consideration <strong>of</strong> technology issues from a range <strong>of</strong>
perspectives (knowledge management).
9.1.1.2 Focus on ITAR
ITAR relates to Section 38 <strong>of</strong> the USA’s Arms Export Control Act (22 USC 2778), which
authorises the President to control the export and import <strong>of</strong> defence articles and defence
services.
According to the ‘Defense Industry Daily’ website [205]
‘<strong>The</strong> problem is that complex ITAR rules force the UK and Australia to wade through a
weeks-long process to get military export approvals, sometimes on mundane weapons
parts and components. <strong>The</strong> restrictions are wide-ranging and extend to “defense
services”, which can include “furnishing <strong>of</strong> assistance, including training, to foreign
persons in the design, engineering, development, production, processing, manufacture,
use, operation, overhaul, repair, maintenance, modification, or reconstruction <strong>of</strong> defense
articles, whether in the United States or abroad” or furnishing <strong>of</strong> technical data.
Unsurprisingly, therefore, this process also plays a role in joint defense projects’.
9.1.2 Armed Forces Health Longitudinal <strong>Technology</strong> Application
<strong>The</strong> US Military Health System (MHS) lost records <strong>of</strong> almost 5,000 patient encounters
because <strong>of</strong> hardware and s<strong>of</strong>tware problems with portions <strong>of</strong> the Defense Department’s
Armed Forces Health Longitudinal <strong>Technology</strong> Application (AHLTA) electronic health
record system [206].
<strong>The</strong> system experienced backup problems with data stored locally at military treatment
facilities (MTFs). <strong>The</strong> s<strong>of</strong>tware problems occurred after local cache servers (LCS) were
installed at 101 facilities and Northrop Grumman provided a new AHLTA s<strong>of</strong>tware
patch. <strong>The</strong> patch, designed to improve LCS performance, was successfully installed at 99
MTFs, but did not work at Fort Stewart, Ga., and Fort Drum, NY, when it was installed.
A database flag or trigger was incorrectly set at those two locations. Consequently,
clinical encounters between doctors and patients were not captured and stored because
the system viewed each as an inactive patient. As a result, 2,608 encounters were not
captured at Fort Drum and another 978 at Fort Stewart. MHS later resolved the patch
problems at those two locations.
A hardware problem at Fort Hood, Texas, in September resulted in the loss <strong>of</strong>
information from 1,400 clinical encounters. That loss was because <strong>of</strong> a hardware failure
in a Redundant Array <strong>of</strong> Independent Disks when a Hewlett-Packard technician installed
a new piece <strong>of</strong> equipment and inadvertently erased all the data on the disk by setting it to
factory default. No backup was in place.
49

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
9.1.2.1 Findings
9.2 Civil
• <strong>The</strong>re was a lack <strong>of</strong> redundancy in the system. <strong>The</strong>re was no backup system in
place and paper records were not kept;
• <strong>The</strong>re was a lack <strong>of</strong> user feedback in the system. Errors were not apparent to the
user until the entire system failed.
9.2.1 <strong>The</strong> Regional Information Systems Plan
<strong>The</strong> Wessex Regional Health Authority’s (WHRA) Regional Information Systems Plan
(RISP), aimed to achieve integration across the health region, and began in the mid
1980’s. <strong>The</strong> plan envisaged the development <strong>of</strong> five core computer systems covering
hospital information, manpower estates, community care and accountancy, operating to
common standards in every district region. Development was to be completed within five
years at an estimated cost <strong>of</strong> £25.8 million (at 1984/1985 prices), with associated revenue
costs over the five-year period <strong>of</strong> £17.5 million.
In April 1990, when the project was <strong>of</strong>ficially abandoned at least £43 million had been
spent. Jeffcott and Johnson [58] analysed the organisational issues involved in the RISP
case and identified the following factors that led to the system’s failure.
9.2.1.1 Findings:
• <strong>The</strong> NHS risk management strategy underestimated the risks involved in the
adoption and implementation <strong>of</strong> new technologies. <strong>The</strong> risk management strategy
was developed in response to increasing financial pressures from litigation for
clinical negligence, but did not consider the risks involved in the introduction <strong>of</strong>
new technologies;
• <strong>The</strong> focus on risk assessment and avoidance <strong>of</strong> litigation can be understood in
historical and organisational terms. In 1975, the cost <strong>of</strong> clinical litigation in the NHS
in England alone was around £1 million; by 1990, survey data suggests the cost had
risen to around £50 million; in 1996 the costs were about £200 million [207], [208];
• Risk assessment is a task that is not well understood in the NHS. <strong>The</strong> NHS
Executive report that it is only carried out ‘because we have to include it in the
business case’ rather than because it is recognised as a central task <strong>of</strong> successful
project planning and management;
• In terms <strong>of</strong> prior experience, the RISP was the first project <strong>of</strong> its kind in the WRHA;
• At the time that RISP was conceived in 1982, there was no clear and agreed national
framework for information management. Until 1986 the Department <strong>of</strong> Health’s
(DoH) IT policy allowed each regional health authority to develop its own IT
services and decide what hardware and s<strong>of</strong>tware to buy, within the framework <strong>of</strong>
broad national guidelines and international rules;
50

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
• <strong>The</strong> role <strong>of</strong> the DoH was to develop and advise on strategic and policy issues and
the performance <strong>of</strong> major tasks. <strong>The</strong>refore RISP should have been recognised as a
high- risk project. It was left to a health authority with no previous experience <strong>of</strong>
such projects. Most <strong>of</strong> the responsibility for technology procurement, project
management and budgeting was left firmly on the WRHA’s shoulders;
• Risk was further exacerbated by the WRHA’s lack <strong>of</strong> consideration <strong>of</strong> qualitative
measures such as experience, past performance and reputation, when awarding their
contracts;
• <strong>The</strong>re were serious conflicts <strong>of</strong> interest at a senior level within the authority and it
was not clear whether those suppliers recruited for the RISP project were the best
available: ‘A Member <strong>of</strong> WRHA who was also a Director <strong>of</strong> IBM, promoted the
Andersen Consulting bid.’ This was because the Andersen Consulting bid involved
the procurement <strong>of</strong> IBM s<strong>of</strong>tware and terminals. <strong>The</strong> appointed auditor for RISP
found no evidence that the region had undertaken a full evaluation <strong>of</strong> the alternative
suppliers.
9.2.2 London Ambulance Service Computer Aided Despatch
<strong>The</strong> main objective <strong>of</strong> the London Ambulance Service Computer Aided Despatch
(LASCAD) project was to automate many <strong>of</strong> the human-intensive processes <strong>of</strong> manual
despatch systems associated with ambulance services in the UK. In the 1990’s the
London Ambulance Service (LAS) invested considerable money in order to implement an
effective Computer Aided Despatch (CAD) project.
LAS scrapped a development by IAL (BT subsidiary) at a cost <strong>of</strong> £7.5 million in October
1990. In June 1991, they signed a £1.1 million contract with Systems Options to provide
a CAD system. This attempt failed in 1992.
<strong>The</strong> system replaced a manual system for taking emergency calls and allocating resources
to incidents. It relied on automatic vehicle tracking and ambulance crews reporting the
status <strong>of</strong> the call using Mobile Data Terminals (MDTs) to allocate resources to incidents.
On the night <strong>of</strong> Monday 26th October to the morning <strong>of</strong> Tuesday 27th October 1992
things started to go wrong at the HQ <strong>of</strong> LAS. It was reported that a flood <strong>of</strong> 999 calls
(some 2900 instead <strong>of</strong> the usual 2300) apparently swamped operators’ screens. Many
recorded calls were being wiped <strong>of</strong>f screens. This, in turn, caused a mass <strong>of</strong> automatic
alerts to be generated indicating that calls to ambulances had not been acknowledged.
<strong>The</strong> report <strong>of</strong> the public inquiry portrays a more complex picture <strong>of</strong> the so-called
technical problems experienced by the LASCAD system than that reported either in the
computing or general press. It is interesting that they conclude: ‘On 26th and 27th
October the computer system did not fail in a technical sense. Response times did on
occasions become unacceptable, but overall the system did what it had been designed to
do. However, much <strong>of</strong> the design had fatal flaws that would, and did, cumulatively lead
to all <strong>of</strong> the symptoms <strong>of</strong> systems failure.’
Various technical and HF issues resulted in communication difficulties that meant the
system was <strong>of</strong>ten unaware <strong>of</strong> the location and status <strong>of</strong> resources. <strong>The</strong> result <strong>of</strong> this was
that the system sometimes made incorrect allocations by:
51

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
• sending multiple ambulances to the same incident;
• sending ambulances that were not the closest ones to the incident;
• not sending an ambulance, which was in fact available to be sent.
This caused frustration to members <strong>of</strong> the public and ambulance crews who telephoned or
radioed the control centre. Slow s<strong>of</strong>tware, user interface problems and understaffing in
the control centre combined to increase telephone and radio traffic and resulted in the
system slowing to unacceptable levels.
On 4 November 1992 the system failed completely due to a programming error and the
back up system, which had not been adequately tested, did not work. Claims were later
made in the press that up to 20–30 people may have died as a result <strong>of</strong> ambulances
arriving too late on the scene. Some ambulances were taking over three hours to answer a
call, whilst the government’s recommended maximum is 17 minutes for inner-city areas.
Arguably the LASCAD project was the most visible UK information systems failure in
recent years.
9.2.2.1 Findings
• <strong>The</strong> LASCAD report [209] states that ‘the size <strong>of</strong> the programme and the speed
and depth <strong>of</strong> change were simply too aggressive for the circumstances.’ <strong>The</strong>
Inquiry Team found that neither the CAD system itself, nor its users, were ready
for full implementation on 26 October 1992.
• <strong>The</strong> CAD s<strong>of</strong>tware was not complete, not properly tuned, and not fully tested. <strong>The</strong>
resilience <strong>of</strong> the hardware under a full load had not been tested. <strong>The</strong> fall back
option to the second file server had not been tested.
• Staff both within Central Ambulance Control (CAC) and ambulance crews had
‘no confidence in the system and had not been fully trained’. <strong>The</strong> physical
changes to the layout <strong>of</strong> the control room on 26 October 1992 meant that CAC
staff were working in unfamiliar positions, without paper backup, and were less
able to work with colleagues with whom they had jointly solved problems before.
Control room staff had little previous experience <strong>of</strong> using computers;
• Satisfactory implementation <strong>of</strong> the system required changes to a number <strong>of</strong>
existing working practices. Senior Management believed that implementation <strong>of</strong>
the system would, in itself, bring about these changes. In fact many staff found it
to be an operational ‘strait jacket’;
• <strong>The</strong>re were a number <strong>of</strong> basic flaws in the CAD system and its supporting
infrastructure. In summary the system and its concept had several major
problems:
• it required near perfect input information from users
• poor interface between crews and the system
• slow response times for certain screen-based activities
• lack <strong>of</strong> robustness <strong>of</strong> the system (including unreliability and system
‘lockups’).
52

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
• ‘Management clearly underestimated the difficulties involved in changing the
deeply ingrained culture <strong>of</strong> the LAS, and as a result the NHS agenda for
LASCAD was far too aggressive’.
• Computing within the NHS is complicated by the fact that no one body has
overall responsibility for IT. IT is exploited and controlled at a number <strong>of</strong>
different levels: region, trust, hospital, department, speciality and general
practice. Each stakeholder has a different perception <strong>of</strong> IT. Region and trust ‘tend
to emphasise administrative systems’. Hospital and General Practitioner (GP)
surgeries ‘emphasise clinical applications’;
• LAS management were under ‘undue pressure’ from the NHS to ensure that the
LASCAD system was implemented on time and within budget. This pressure and
a fear <strong>of</strong> failure ‘blinded them to some <strong>of</strong> the fundamental difficulties <strong>of</strong> the
system implementation’.
• LAS management chose an inappropriate supplier, a decision that was seen as
being ‘detrimental to the project’: ‘In awarding the contract for CAD to a small
s<strong>of</strong>tware house, with no previous experience in similar emergency service
systems, LAS management was taking a high risk’;
• <strong>The</strong>re were also concerns with the quality <strong>of</strong> the contractor that had been recruited
to build the CAD system. ‘<strong>The</strong> LAS board gave a misleading impression, by the
project team, <strong>of</strong> the previous experience <strong>of</strong> the lead contractor in emergency
service systems’;
• <strong>The</strong> original procurement document, which was drafted within the guidelines
provided by the regional health authority, put price before quality.
• <strong>The</strong> reports found that whilst the South West Thames Regional Health Authority
procurement rules ‘emphasised open tendering and the quantitative aspects <strong>of</strong>
procurement (obtaining the best price) over qualitative aspects (i.e. who will do
the job best?)’ and the successful contractor substantially underbid an established
supplier and were put under pressure to complete the system quickly.
9.3 Discussion <strong>of</strong> Case Studies
<strong>The</strong>se case studies show that the failure <strong>of</strong> technology programmes can occur for a
variety <strong>of</strong> different reasons, indeed failure is <strong>of</strong>ten the result <strong>of</strong> several factors interacting
with one another. It is also apparent that the failure <strong>of</strong> technology is not simply the result
<strong>of</strong> insurmountable technical problems. As also suggested in the literature there are a
number <strong>of</strong> organisational and contextual factors that can influence the likelihood <strong>of</strong>
success or failure. It is clear that in some cases the selection <strong>of</strong> technology has not
considered these factors sufficiently. <strong>The</strong>se issues are perhaps easier to overcome than
the cases where ingrained organisational culture, politics or structure has contributed
strongly to the eventual failure. <strong>The</strong> over-riding impression is that successfully
integrating a new technology into organisations is not easily done.
53

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
10 Managing <strong>Technology</strong> <strong>Insertion</strong>
10.1 Methods <strong>of</strong> Supporting <strong>Technology</strong> <strong>Insertion</strong>
Given that organisational, social and political aspects <strong>of</strong> organisations have been shown
to have such an influence on the success or failure <strong>of</strong> technology insertion projects, it
would seem appropriate for practitioners to consider some <strong>of</strong> the techniques available to
help manage these aspects. As we have seen, the evidence suggests that most
practitioners ‘muddle through’, or use the ‘task and technology’ approach. Despite this,
socio-technical systems methods do exist that have been developed with the aim <strong>of</strong>
supporting practitioners in the identification and management <strong>of</strong> socio-technical risks.
10.1.1 Open Systems Task Analysis
Eason and Harker’s Open Systems Task Analysis (OSTA) follows a socio-technical
systems analysis model in which technical requirements are specified alongside social
systems requirements, such as usability and acceptability [210]. Its underlying aim is to
provide a methodology for understanding the transformation that occurs when a computer
system is introduced into the organisation.
10.1.2 Personnel and Organisational Implications <strong>of</strong> New <strong>Technology</strong>
A methodology known as POINT (Personnel and Organisational Implications <strong>of</strong> New
<strong>Technology</strong>) has been developed by the Centre for Human Sciences (CHS) under a MoD
Corporate Research Programme. This technique aims to identify the likely future
implications <strong>of</strong> new technologies in terms <strong>of</strong> personnel and organisational outcomes
within the military, including implications for new skills requirements, role changes,
impacts upon team, branch and organisational structures. Taking a socio-technical
systems approach, this method involves the use <strong>of</strong> interviews with subject matter experts,
workshops and the development and specification <strong>of</strong> technology options.
<strong>The</strong> POINT methodology has been used in several cases in order to generate predictions
<strong>of</strong> the impacts that new and future technologies will have upon their military
organisations. This method has been used in several technology programmes in order to
generate likely personnel and organisational consequences <strong>of</strong> new technologies. For one
such technology programme, predicted impacts at the equipment level included the
following:
• Automation will remove many lower level technical and monitoring tasks;
• New requirements for specialist IT and s<strong>of</strong>tware engineering skills;
• Possible amalgamation <strong>of</strong> traditional operator and technician roles.
At the management level, predicted implications included:
• need for comprehensive IT and Computer and Information Science (CIS)
knowledge;
• lower management roles to assume local network management;
54

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
• possible convergence <strong>of</strong> the roles <strong>of</strong> ‘Yeoman’ and ‘Foreman’.
Personnel and organisational issues that needed to be addressed included:
• current organisational structures need to be assessed to take account <strong>of</strong> role
changes;
• provision needs to be made for new specialist IT and s<strong>of</strong>tware engineering roles;
• policies on recruitment and retention need revision to reflect future needs.
Authors such as Martin and McLaughlin [211], and Bowyer and Martin [212] claim that
the POINTS method has been successfully validated through application in these
technology programmes. Despite the face validity <strong>of</strong> some <strong>of</strong> the POINTS outputs, it is
unclear from the literature whether predictions have proven accurate. Similarly there is
little evidence <strong>of</strong> whether this approach has enabled technology programmes to avoid the
risks that were identified.
10.1.3 Discussion <strong>of</strong> OSTA and POINTS Methods
Socio-technical design approaches focus on involving users in the design process and on
considering a wide range <strong>of</strong> social and technical alternatives. Although these approaches
involve users at various stages <strong>of</strong> the design, they are not without their problems. One <strong>of</strong>
the main problems is the need for a socio-technical expert to guide the design process and
support users. Another is the degree to which it can be integrated with other system
development processes and methods. This need for extra effort needs to be understood.
Another problem is that these methods can only be used if the organisational and political
climate accepts practitioners, their requests for workshops and views these methods as
valuable and valid processes. If these methods are not perceived as valuable, their costs
may be perceived to be unnecessary by programme managers. Sometimes the issue <strong>of</strong>
cost effectiveness is raised and there must be a commitment by management to involve
users and take on board their requirements. <strong>The</strong> OSTA method has been criticised by
some due to the fact that whilst it provides useful guidance it is difficult to apply
effectively.
10.2 Tools and S<strong>of</strong>tware to Support Practitioners
10.2.1 Supporting <strong>Technology</strong> <strong>Insertion</strong> Programmes
In recent years, a number <strong>of</strong> tools have been developed to support practitioners in
decision making about when technology insertion programmes should be carried out. In
platforms with long lifespans, component obsolescence is an issue that needs careful
consideration. <strong>Technology</strong> insertion <strong>of</strong> new components can have an impact on system
performance and can be a costly process to undergo. In order to help the practitioner
predict when technology insertion is appropriate within the platform’s lifecycle, a number
<strong>of</strong> tools have been developed, which are described in the following sections.
10.2.1.1 Mitigation <strong>of</strong> Obsolescence Cost Analysis (MOCA)
<strong>The</strong> Centre for Advanced Life Cycle Engineering (CALCE) in the Dept <strong>of</strong> Mechanical
Engineering at the University <strong>of</strong> Maryland, has developed a methodology for determining
55

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
component obsolescence and the associated impact on costs for long field life electronic
systems. <strong>The</strong> Mitigation <strong>of</strong> Obsolescence Cost Analysis (MOCA) is a s<strong>of</strong>tware tool
designed to support proactive design and life cycle planning <strong>of</strong> systems. <strong>The</strong> MOCA
developers argue [213] that methodologies are needed to address optimal design in order
to minimise the cost <strong>of</strong> component obsolescence and technology insertion. This tool is
designed to support decisions on:
• when best to carry out design refresh programmes;
• what obsolete system components should be replaced at a specific design refresh
(versus continuing with some other obsolescence mitigation strategy);
• what non-obsolete system components should be replaced at a design refresh.
Based on a detailed cost analysis model, the methodology determines the optimum design
refresh plan during the life <strong>of</strong> the product. <strong>The</strong> design refresh plan consists <strong>of</strong> the number
<strong>of</strong> design refresh activities, their respective calendar dates and content necessary to
minimise the life cycle sustainment cost <strong>of</strong> the product. <strong>The</strong> decisions that govern
whether a technology is changed (replaced or upgraded) or not changed at a design
refresh depend on the obsolescence attributes <strong>of</strong> the specific technology and on the
‘utility’ to the system realised by changing the technology (economic, performance, and
reliability) [214]. To make the decision in a coupled-technology process, the MOCA
developers formulated Bayesian Belief Networks (BBNs). BBNs are applicable for
reasoning about beliefs under conditions <strong>of</strong> uncertainty and using disparate sources <strong>of</strong>
evidence (diverse data sources, including subjective beliefs and when all <strong>of</strong> the data
entering into the decision is highly uncertain). An example <strong>of</strong> this is shown in Figure 8.
Figure 8: Example Bayesian Belief Network (BBN) for determining whether to
replace electronic components at a specific redesign point [218].
56

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
One advantage <strong>of</strong> using BBNs is that sharing an understanding among many
heterogeneous stakeholders (procurement, design, manufacturing, etc.) using both
qualitative and explicit data is a possibility. <strong>The</strong> approach followed in this work is to use
decision analysis to decide design refresh content for candidate design refresh dates
generated by the MOCA tool. This approach enables the consideration <strong>of</strong> all the decisions
affecting variables at the same time. Aspects such as the availability <strong>of</strong> a new part to
replace the old part, the available stock <strong>of</strong> the old part, the performance and reliability
change due to part changes, impacts on the system s<strong>of</strong>tware due to hardware changes, requalification
that may be triggered by part changes, etc can be incorporated. This
approach ensures that users consider both dimensions <strong>of</strong> optimisation, i.e., date <strong>of</strong> the
design refresh along with what is changed at the design refresh. <strong>The</strong> data associated with
utility nodes represent the inputs from various stakeholders, i.e., ‘customer-directed
value’.
Other tools which have been developed to support practitioners in technical aspects <strong>of</strong>
technology insertion include:
10.2.1.2 Simulation Assisted Reliability Assessment (SARA)
Simulation Assisted Reliability Assessment (SARA) is another CALCE approach to
supporting technology insertion practitioners. SARA has been developed to use ‘physics
<strong>of</strong> failure’ based principles and s<strong>of</strong>tware to assess whether a part or system can meet
defined life cycle requirements [215]. This tool takes into account technical component
requirements based on materials, geometry, and operating characteristics. <strong>The</strong> SARA
s<strong>of</strong>tware ‘can be used to assess life expectancy <strong>of</strong> electronic hardware under anticipated
life cycle loading conditions, as well as under accelerated stress test conditions’. <strong>The</strong>
SARA s<strong>of</strong>tware toolkit is designed to help engineers predict and manage technology
component reliability in technology insertion programmes. This s<strong>of</strong>tware does not
consider organisational or human factors that influence design decisions.
10.2.1.3 Concept Analysis and Design Evaluation Toolkit (CADET)
Numerous spacecraft technology development programs are sponsored by the DoD and
NASA with the goal <strong>of</strong> enhancing spacecraft performance, reducing mass, and reducing
cost. Bearden et al [216] found that it is <strong>of</strong>ten the case that technology programs, in the
interest <strong>of</strong> maximising subsystem-level performance and/or mass reduction, do not
anticipate synergistic system-level effects.
To address these issues, a concept analysis methodology and s<strong>of</strong>tware toolset was
developed. <strong>The</strong>se tools, collectively referred to as the concept analysis and design
evaluation toolkit (CADET), were developed in order to try to facilitate technology
selection, and identify associated effects on cost, risk and performance at the system and
subsystem level. CADET claims to enable:
(1) quick response to technical design and cost questions
(2) assessment <strong>of</strong> the cost and performance impacts <strong>of</strong> existing and new
designs/technologies
(3) estimation <strong>of</strong> cost uncertainties and risks. <strong>The</strong>se capabilities aid mission
designers in determining the configuration <strong>of</strong> remote sensing missions that
meet essential requirements in a cost-effective manner.
57

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
<strong>The</strong>se tools focus on technical and cost aspects <strong>of</strong> technology insertion decision making,
but do not consider wider organisational implications.
10.2.1.4 Aviation System Risk Model (ASRM)
<strong>The</strong> Aviation System Risk Model (ASRM), a Probabilistic Decision Support System
(PDSS), is being developed [217] to assist NASA program managers in evaluating new
technologies that are intended to lower the fatal aircraft accident rate. This decision
support system utilises the method <strong>of</strong> case-based reasoning in which case studies <strong>of</strong>
specific accidents are analysed and modelled to assess reduction through the insertion <strong>of</strong>
appropriate technologies.
<strong>The</strong> analytical approach used in this research is a systematic method for modelling
aircraft accidents and assessing risk reduction. <strong>The</strong> analytic modelling approach consists
<strong>of</strong> the following stages:
[1] Describing a case-based scenario and determining the events involved in the
occurrence <strong>of</strong> the accident/incident.
[2] Identifying the causal factors (nodes) present in these events using the Human
Factors Analysis and Classification System (HFACS) taxonomy.
[3] Construction <strong>of</strong> an influence diagram depicting the interrelationships among
these nodes.
[4] Building a BBN propagated with conditional probabilities. Probabilities are
based on a combination <strong>of</strong> statistical frequencies from the National
Transportation Safety Board (NTSB) database and judgements from subject
matter experts.
[5] Inserting the relevant technologies/interventions into the model <strong>of</strong> the
particular case, attaching each to a causal factor.
[6] Assessing the relative risk <strong>of</strong> the particular case and the reduction in risk
resulting from varying combinations <strong>of</strong> technology insertions.
<strong>The</strong> following model (see Figure 9) depicts the causal factors identified in an accident
case study (Air Ontario 1363 [218]) and the interactions among them.
58

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
Figure 9: ASRM Screenshot
This approach allows absolute percentage risk decrease and relative percentage risk
decrease to be calculated when;
• there are no technology insertions;
• following the insertion <strong>of</strong> all relevant technologies;
• when new technologies are introduced separately.
This model is designed to depict causal factors, by combining statistical data with
judgement <strong>of</strong> subject matter experts to obtain estimates <strong>of</strong> probabilities. <strong>The</strong> authors state
that the ‘Air Ontario 1363 case study demonstrates the importance <strong>of</strong> considering not
only errors made by pilots and the cabin crew but also situational (e.g. weather),
supervisory, and organisational factors’.
10.2.1.5 Socio-technical Team-working for OR Modelling (STORM)
STORM is an algorithm developed by DSTL which combines Tuckman’s team maturity
process [219] with Noble’s [220] theory <strong>of</strong> knowledge enablers to create a model <strong>of</strong> the
social and cultural characteristics <strong>of</strong> teams. STORM is described as an ‘attempt to use
modelling and simulation to examine some <strong>of</strong> the human issues that may emerge in the
implementation <strong>of</strong> the concept <strong>of</strong> agile mission grouping’.
59

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
STORM integrates Noble’s and Tuckman’s processes to model the dynamic development
<strong>of</strong> teams from first formation to full maturity. As a representation <strong>of</strong> the impacts <strong>of</strong> social
and cultural factors on team performance in a coalition NEC context, STORM allows
informal team relationships to be modelled and can provide insight into how NEC can
influence team behaviour. This tool was developed in response to a need for a model
capable <strong>of</strong> dealing with agile, ad hoc team formation associated with Agile Mission
Grouping [221], [222]. This model focuses on team performance based on team
composition, context and maturity.
STORM currently does not model the influences <strong>of</strong> technology directly; however, aspects
<strong>of</strong> co-location versus remote computer mediated communication are modelled. Work is
currently being undertaken to try to model team acceptance <strong>of</strong> technology and this will be
an interesting development. <strong>The</strong>re is potential to develop a similar model, or collaborate
with DSTL to look at how technology influences interact.
Figure 10: Interacting Nodes in the ‘STORM’ Model
10.3 Summary <strong>of</strong> the Extant TI Tools
A number <strong>of</strong> tools, methods and techniques have been, or are being, developed to support
practitioners during the technology insertion process. Whilst these options are worth
considering, many <strong>of</strong> them are either not fully developed, or are narrow in their scope.
This is especially true <strong>of</strong> the cited s<strong>of</strong>tware tools and models that focus exclusively on
engineering and cost-based aspects <strong>of</strong> TI decision making. Whilst this is undoubtedly a
valuable goal, there are far fewer tools for supporting practitioners in understanding and
measuring organisational implications <strong>of</strong> technology decisions. OSTA and POINTS may
be valuable methodologies, however, it appears that there are no discrete tools to guide
the novice practitioner. Such guidance may be particularly relevant given that these
methods rely upon the skills and experience <strong>of</strong> the implementer in order to be successful.
60

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
<strong>The</strong>se tools and techniques do however lead to the conclusion that a toolset may be a
worthwhile and feasible goal for consideration. <strong>The</strong> aviation system risk model and the
STORM model indicate that decision support tools might be a worthwhile focus <strong>of</strong>
attention for a technology insertion toolset.
61

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
11 Gaps in <strong>Technology</strong> <strong>Insertion</strong> Capabilities
11.1 Gaps in Understanding, Methods and Tools
This scoping study has shown that there are a great many risks to technology insertion
programmes. Despite the developments in risk assessment tools and techniques, research
has found that the rate <strong>of</strong> technology project failure has not reduced. <strong>The</strong> case study
evidence suggests that risk assessment tools are sometimes applied to these projects, but
assessors <strong>of</strong>ten take a narrow view <strong>of</strong> the risks. This narrow view <strong>of</strong> risk can be the result
<strong>of</strong> organisational culture. NHS technology projects for example, have been found to focus
on avoiding litigation and financial risk at the expense <strong>of</strong> the consideration <strong>of</strong> other
potential risks.
In the context <strong>of</strong> technology insertion it is clear that the number <strong>of</strong> risks, at
organisational, technical and systems levels, coupled with the <strong>of</strong>ten overly optimistic
expectations <strong>of</strong> project backers, can mean these programmes become very challenging
and stressful to manage.
<strong>The</strong>re is strong evidence indicating that a technocentric viewpoint exists within many
programmes, where new technology is given a high value, and its introduction seen as a
suitable end in itself. This view can mean that individual technical issues are well
examined and understood, whereas contextual factors surrounding technology use end up
being ignored. Technocentric culture can mean that important factors such as the
organisational, contextual, political and the human may be less well understood and
therefore either not considered at all, or not well managed.
Even when technical issues are well understood there is a risk that the impacts <strong>of</strong>
technology insertion on the established system may not have been fully considered. In
isolation the insertion <strong>of</strong> new components, such as an air conditioning unit in a ground
vehicle, seems like it provides substantial benefits. It is clear however that changes such
as this can affect the performance <strong>of</strong> the system in unexpected ways. In this example the
air conditioning unit may take up more space in the vehicle, leaving less space for
personnel. Additionally, the unit may require greater power, and add weight to the
system, which will inevitably lead to a reduced speed capability and increased fuel
consumption.
It is precisely because <strong>of</strong> this prediction problem that so many practitioners struggle with
the challenge <strong>of</strong> introducing new technologies into a platform, system or organisation. As
we have seen there are a great many academic theories surrounding technology and its
impacts, however there is very little practical guidance for the practitioner. For those
decision makers who do not have access to outside expertise there is little in the any <strong>of</strong>
tools or guidance to provide support. As we have seen, knowledge management resources
may provide information about other practitioners’ experiences, however this does not
guarantee that these experiences will be applicable to the practitioner’s current needs.
<strong>The</strong>re is a good chance that ‘knowledge’ in these resources will be based on individual
experiences <strong>of</strong> technical problems rather than on insights into the socio-technical or
human factors issues that surrounded these events. Alternative approaches may prove to
be a valuable investment <strong>of</strong> effort if we are to better support decision makers and
practitioners.
62

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
11.2 Identifying and Training the <strong>Technology</strong> <strong>Insertion</strong>
Practitioners
<strong>The</strong> literature, case studies and SME evidence suggest that technology insertion risks
occur at various stages <strong>of</strong> any technology insertion programme. Engineers selecting
components to upgrade military platforms may be familiar with selecting equipment
based on capability and cost, but must also consider the implications that changes may
have on overall platform performance. How important is component performance when
compared with its ruggedness, international restrictions, reliability or number <strong>of</strong>
suppliers?
At a higher level <strong>of</strong> decision maker, there are risks in selecting the wrong technologies
and platforms to develop. Are major platforms relevant for asymmetrical warfare? Will a
new IT system be compatible with existing systems or coalition systems?
<strong>Technology</strong> insertion decisions can have implications for the number and type <strong>of</strong>
personnel needed in the organisation as well as for recruitment and retention policies.
This scoping study suggests that in terms <strong>of</strong> the military, TI practitioners include:
• Customers;
• MoD Integrated Project Teams (IPT);
• MoD DECs (Director <strong>of</strong> Equipment Capability);
• Contractors;
• Training Organisations.
Given the number <strong>of</strong> people responsible for TI and the uncertainty and risks surrounding
TI decisions, it is clear that TI practitioners need either training, guidance in selecting
effective techniques or support in decision making. Weaknesses in the acquisition process
have been identified in this report, but it is beyond the scope <strong>of</strong> this study to recommend
changes to this aspect <strong>of</strong> military organisations.
11.3 Are there any common lessons?
Given the challenges in managing technology insertion programmes, it seems likely that
practitioners need to be well trained in identifying and managing project risks.
Practitioners may need to be trained to take a systems view <strong>of</strong> technology projects, so
they are able to not only identify technology capabilities, but also assess overall system
impact, and identify organisational risks, before they commit to insertion decisions.
It is also apparent that the tools available to support practitioners are lacking in efficacy.
If risk assessment tools are failing to help practitioners identify the key risks, then it
seems reasonable to suggest that more effective or task-appropriate tools are required. So,
to summarise:
• Practitioners may not always have experience <strong>of</strong> the likely risks <strong>of</strong> technology
insertion or the impacts <strong>of</strong> these new technologies upon the user and
organisation. This issue is especially likely where the impacts occur beyond
the practitioners’ area <strong>of</strong> expertise, e.g. impacts on productivity, morale,
63

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
organisational structure. Practitioners may need to be trained to take a systems
view and to understand the importance <strong>of</strong> domain experts, such as human
factors pr<strong>of</strong>essionals, who are required to identify likely impact and
organisational risks throughout project lifecycles.
• Currently available tools lack effectiveness. Risk assessment tools do not help
with identification <strong>of</strong> identify unexpected risk, nor do they take into account
changing project conditions. More effective, task appropriate tools are
required to support practitioners.
11.4 Can we predict whether TI will succeed or fail?
One <strong>of</strong> the questions this scoping study sought to answer was whether or not it is possible
to predict whether insertion <strong>of</strong> a technology will succeed or fail. <strong>The</strong> answer is that it is
probably not possible to predict success or failure outright, for the simple reason that
there are too many interacting variables during technology insertion within organisations.
Similarly it is difficult to envisage being able to predict the impact that a new technology
will have upon an organisation. As Eason put it: ‘computing technology has been a major
force for change in organisations for over 30 years and throughout that time there has
been little evidence that developers <strong>of</strong> technologies are able to predict or plan
organisational outcomes’ [24].
Perhaps a more valuable focus <strong>of</strong> effort would be upon attempting to develop better tools
to help identify and manage the many risks to technology insertion projects.
11.5 A Toolkit to Support Practitioners
A review <strong>of</strong> the toolkits that have been developed to support practitioners in managing
technology insertion programmes indicates that several approaches may be feasible.
<strong>The</strong>re are two major challenges to technology insertion practitioners that could be
supported by a toolkit. <strong>The</strong> first is aiding the identification and management <strong>of</strong> risks and
the second is in supporting decision making under uncertainty.
11.5.1 Identifying Risks
It is the opinion <strong>of</strong> the author, based on the literature reviewed for this study, that existing
risk assessment and management tools do not provide sufficient support to technology
insertion practitioners in identifying risks. Although existing tools are designed to
support the identification <strong>of</strong> all possible risks, they currently allow users to ignore key
organisational, contextual and systemic factors. It seems that these tools rely heavily on
the practitioner’s insight and experience. Indeed, research has shown that risk assessment
tools support users in identifying and quantifying risks that they are already aware <strong>of</strong>, but
do not effectively support the identification <strong>of</strong> unexpected issues. A toolkit designed to
alert the practitioner to a broader range <strong>of</strong> issues may be a valuable development in
supporting practitioners.
Most standard texts on risk propose that you decompose risks into two components:
• Probability (or likelihood) <strong>of</strong> the risk and
64

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
• <strong>Impact</strong>/loss the risk can cause
<strong>The</strong> next stage is then to define risk as the measure:
Risk = Probability x <strong>Impact</strong>
This standard ‘utility’ measure <strong>of</strong> risk ‘is quite useful for prioritising risks (the bigger the
number the ‘greater’ the risk)’ but according to Fenton et al [223] ‘it’s normally
meaningless’. More importantly, you ‘cannot get the numbers you need to calculate it’.
Fenton goes on to say that the problems with this type <strong>of</strong> risk number is that ‘we cannot
get the Probability number because the probability <strong>of</strong> any risk is conditional on a number
<strong>of</strong> other control events and trigger events. It makes no sense to assign a direct probability
without considering the events it is conditional on. In general it makes no sense (and
would in any case be too difficult) for a risk manager to give the unconditional
probability <strong>of</strong> every ‘risk’ irrespective <strong>of</strong> relevant controls, triggers and mitigants’.
Fenton et al also argue that it does not give a ‘useful <strong>Impact</strong> number as we cannot say
what the impact is without considering the possible mitigating events Risk score is
meaningless. It does not tell us what we really need to know.’ <strong>The</strong> argument is that the
rational way to think <strong>of</strong> risks is ‘in terms <strong>of</strong> causal models (risk maps) with trigger events,
control events, risk events, mitigant events and consequence events.’
Coincidentally, causal models <strong>of</strong> risk can also be used to support decision making under
uncertainty and it is this combination <strong>of</strong> risk assessment and decision support that we turn
to next in order to consider their potential application to TI problems.
11.5.2 Decision Support
Given the number <strong>of</strong> factors that influence whether a technology project is a success,
many technology insertion decisions, even those apparently straightforward ones, are
difficult to make with any degree <strong>of</strong> confidence. We have also seen the implications that
poor quality, inexperienced, uninformed or simply unfortunate technology decisions can
have upon organisations, individuals and systems. Consequently, it seems reasonable to
suggest that the development <strong>of</strong> a decision support toolkit would be a valuable investment
<strong>of</strong> research time and effort.
<strong>The</strong>re is considerable research in the area <strong>of</strong> decision making under uncertainty, and
numerous tools have been developed to support decision makers [223]. <strong>The</strong>se tools have
primarily been developed because, as we have discussed, humans are not perfect
probabilistic decision makers, as they are influenced heavily by their own experiences
and biases. Probabilistic decision-making techniques such as Bayesian networks, have
been found to be useful in supporting people to make difficult decisions, especially where
there is a degree <strong>of</strong> uncertainty over the best course <strong>of</strong> action. <strong>The</strong>re is a wealth <strong>of</strong>
literature within Psychology and Human Factors suggesting that the quality <strong>of</strong> decision
making can diminish under time pressure and stress, two aspects which are clearly
evident in many technology insertion programmes. Literature also finds that under stress
decision makers will focus their attention very narrowly on the information that first
comes to hand. Such natural human responses serve a useful purpose in fight or flight
survival situations, but unfortunately do not always help effective decision making. <strong>The</strong>
provision <strong>of</strong> an effective decision support tool would encourage the evaluation <strong>of</strong>
different options, which may not have otherwise been considered and reduce the risk <strong>of</strong>
key issues being ignored.
65

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
12.1 Summary and Follow on
12 Direction <strong>of</strong> Future Work
This scoping study has shown that there are many risks to technology insertion projects,
which are both difficult to identify and a challenge to manage. <strong>The</strong> study has also found
that whilst technology does impact upon organisations, the influence is neither
deterministic nor unidirectional. Organisation, culture and a host <strong>of</strong> other factors such as
context, also influence technology outcomes.
<strong>The</strong> findings from this study suggest that a probabilistic risk identification toolkit could
have the potential to support uncertain and high-risk technology insertion decisions. <strong>The</strong>
existence <strong>of</strong> basic s<strong>of</strong>tware tools to support practitioners in aspects <strong>of</strong> technology
insertion programmes suggest that developing a toolkit would be a feasible goal. It is
suggested that follow-on work from this task focuses on developing requirements and
methods to guide the development <strong>of</strong> a toolkit to support technology insertion
practitioners.
66

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
13 References
[1] <strong>Technology</strong> <strong>Insertion</strong> Major Project Area. [Online]. Available from:
http://www.timpa.co.uk/ [cited Dec 2006].
[2] Mills, V. (2002). <strong>The</strong> Effects <strong>of</strong> Information <strong>Technology</strong> on Intra-Human
Communication in the Workplace. DSTO Systems Sciences Laboratory.
[3] Wikipedia. Article on the Industrial Revolution. [Online]. Available from:
http://en.wikipedia.org/wiki/Industrial_Revolution [cited Dec 2006].
[4] Arnold, J., Robertson, I.T., and Cooper, C.L. (1995). Work psychology:
understanding human behaviour in the workplace. Pitman, London.
[5] Wall, T.D. (1987). New <strong>Technology</strong> and Job Redesign. In P. Warr Psychology at
Work. 3 rd edn. Penguin, Harmondsworth.
[6] Clegg, C. (1994). Psychology and Information <strong>Technology</strong>: <strong>The</strong> study <strong>of</strong>
cognition in organisations. British Journal <strong>of</strong> Psychology.85, 449-447.
[7] Brynjolfsson, E., Malone, T.W., Gurbaxani, V. and Kambil, A. (1994). An
Empirical Analysis <strong>of</strong> the Relationship Between Information <strong>Technology</strong> and Firm
Size. Management Science.
[8] Moore’s Law on Chips Marks 40 th . BBC News. Monday, 18 April, 2005. [Online]
Available from:
http://news.bbc.co.uk/1/hi/technology/4446285.stm [cited Dec 2006].
[9] Gurbaxani, V. and Mendelson, H. (1990). An Integrative Model <strong>of</strong> Information
Systems Spending Growth, Information Systems Research, 1, 23-46.
[10] Ives, B. and Jarvenpaa, S.L. (1991). Applications <strong>of</strong> Global Information
<strong>Technology</strong>: Key Issues for Management. MIS Quarterly, March 1991, 32-49.
[11] Bartlett, C.A. and Ghoshal, S. (1989). Managing Across Borders. <strong>The</strong>
Transnational Solution, Harvard Business School Press, Boston.
[12] DSTL Knowledge Services. Unclassified Briefing Paper on RAO Human
Capability Priority Research Area 2: <strong>Technology</strong> <strong>Insertion</strong>.
[13] Clegg, C., Coleman, P., Hornby, P., MacLaren, R. and Robson, J. (1996). Tools to
incorporate some psychological and organisational issues during the development <strong>of</strong>
computer ergonomics. Ergonomics, 39, 482-511.
[14] Moynihan, T. (1997). How experienced project managers assess risk. S<strong>of</strong>tware,
IEEE, 14(3)., 35-41.
[15] Lyytinen, K. and Hirschheim, R. (1988). Information systems failures — a survey
and classification <strong>of</strong> the empirical literature. Oxford Surveys in Information
<strong>Technology</strong>.
67

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[16] Hochstrasser, B. and Griffiths C. (1991). Investment Strategy and Management.
IT Controlling. Chapman and Hall, London.
[17] KPMG International. (1999). Project Risk Management: Information Risk
management document in Jeffcott, M. and Johnson, C. (2005). <strong>The</strong> use <strong>of</strong> a
formalised risk model in NHS information system development.
[18] Kemerer, C.F. and Sosa G.L. (1990). Systems Development Risks in Strategic
Information Systems. Information & S<strong>of</strong>tware <strong>Technology</strong>, 33, 212-223, Butterworth-
Heinemann Ltd.
[19] Ewusi-Mensah, K. and Przasnyski, Z. (1994). Factors contributing to one
abandonment <strong>of</strong> information systems development projects. Journal <strong>of</strong> Information
<strong>Technology</strong>, 9, 185-201.
[20] Markus, M.L. and Keil, M. (1994). If we build it they will come: Designing
information systems that users want to use. Sloan Management Review, 35, 11-25.
[21] Jones, C. (1995). Patterns <strong>of</strong> S<strong>of</strong>tware System Failure and Success. London:
International Thomson Computer Press.
[22] Willcocks, L. and Lester, S. (1991). Information systems investments: Evaluation
at the feasibility stage <strong>of</strong> projects. Technovation. 11, 283-302.
[23] Nulden, U. (1996). Failing Projects: Harder to Abandon Than to Continue,
Journal de Projectique, Bayonne, Saint-San~bastien, France, 24-26 Octobre, 63-73.
[24] Eason, K. (2001). Changing perspectives on the organisational consequences <strong>of</strong>
information technology. Behaviour & Information <strong>Technology</strong>, 20, 323-328.
[25] Gibbs, W. (1994). S<strong>of</strong>tware’s Chronic Crisis. Scientific American, 271:33, 72-81.
[26] Keil, M., Mixon, R., Saarinen, T. and Tuunainen, V. (1995). Understanding
Runaway Information <strong>Technology</strong> Projects: Results from an International Research.
Journal <strong>of</strong> Management Information System,.11, 67-97.
[27] (<strong>The</strong> Standish Group International, 1994). Chaos: charting the seas <strong>of</strong>
information technology, a special compass report. . [Online] Available from:
http://standishgroup.com/sample_research/chaos_1994_1.php [Cited Dec 2006]
[28] Legris, P., Ingham, J. and Collerette, P. (2003). Why do people use information
technology? A critical review <strong>of</strong> the technology acceptance model. Information and
Management. 40, 191-204.
[29] McFarlan, W. (1974). Portfolio Approach to Information Systems. Harvard
Business Review, Jan/Feb, 142-150.
[30] Earl, M.J. (1989). Management Strategies for Information <strong>Technology</strong>. Prentice
Hall, Hemel Hempstead.
[31] Corder, C. (1989). Taming your Company Computer. McGraw-Hill, London.
68

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[32] Cash, J., McFarlan, F.W., McKenney, J. and Applegate, L. (1992). A Portfolio
Approach to IT Development. Information Systems Management, Irwin Publishing,
Third Edition.
[33] Parker, M., Benson R., and Trainor, E. (1998). Information Economics: Linking
Business Performance to Information <strong>Technology</strong>. Prentice-Hall, New Jersey.
[34] Willcocks, L. and Griffiths C. (1997). Management and Risk in Major
Information <strong>Technology</strong> Projects. In Willcocks et al (eds). Managing IT as a Strategic
Resource. McGraw-Hill, London, 203-237.
[35] Appleton, D. (1991). Very Large Projects. Datamation, 63-70, 15 January.
[36] Collingridge, D. (1992). <strong>The</strong> Management <strong>of</strong> Scale.London, Routledge.
[37] Davidson, F. and Huot, J.C. (1991). Large-scale Projects: Management Trends for
Major Projects. Cost Engineering; 33, 15-23.
[38] Morris, P. and Hough, G. (1987). <strong>The</strong> Anatomy <strong>of</strong> Major Projects. Chichester,
John Wiley and Sons.
[39] Willcocks, L. and Griffiths, C. (1994). Predicting Risk <strong>of</strong> Failure in Large-Scale
Information <strong>Technology</strong> Projects. Technological Forecasting and Social Change, 47,
205-228.
[40] Miller, D. (1983). <strong>The</strong> Correlates <strong>of</strong> Entrepreneurship in Three Types <strong>of</strong> Firms.
Management Science, 29, 770-791.
[41] Hertzum, M. (2002). Organisational Implementation: A Complex but
Underrecognised Aspect <strong>of</strong> Information-System Design In O.W. Bertelsen, S.
Bødker, and K. Kuutti (eds.), NordiCHI 2002: Proceedings <strong>of</strong> the Second Nordic
Conference on Human-Computer Interaction (Aarhus, Denmark, October 19-23,
2002), 199-202. ACM Press, New York.
[42] Curran, C. and Connally, M. (2001). A Value-centric Perspective for Managing
Information <strong>Technology</strong>. Viewpoint Whitepaper. [Online] Available from:
http://www.diamondcluster.com/ideas/viewpoint/Viewpointv2n1.asp [cited Dec 2006].
[43] Abdel-Hamid, T.K. (1988). Understanding the ‘90 % Syndrome’ in S<strong>of</strong>tware
Project Management. Journal <strong>of</strong> Systems and S<strong>of</strong>tware, 8, 319-330.
[44] Abdel-Hamid, T.K. and Madnick, S.E. (1991). S<strong>of</strong>tware Project Dynamics: An
Integrated Approach, Prentice Hall, Englewood Cliffs, NJ.
[45] Brooks, F.P. (1975). <strong>The</strong> Mythical Man-Month: Essays <strong>of</strong> S<strong>of</strong>tware Engineering,
Addison-Wesley, Reading, MA.
[46] DeMarco, T. (1982). Controlling S<strong>of</strong>tware Projects, Yourdon, New York.
[47] Zmud, R. (1980). Management <strong>of</strong> Large S<strong>of</strong>tware Efforts. MIS Quarterly, 4, 45-
55.
69

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[48] Keil, M. and Mann, J (1997). <strong>The</strong> Nature and Extent <strong>of</strong> Information <strong>Technology</strong>
Project Escalation: Results from a Survey <strong>of</strong> IS Audit and Control Pr<strong>of</strong>essionals. IS
Audit & Control Journal (1), 40-48.
[49] Brockner, J. (1992). <strong>The</strong> Escalation <strong>of</strong> Commitment to a Failing Course <strong>of</strong>
Action: Toward <strong>The</strong>oretical Progress. Academy <strong>of</strong> Management Review (17:1),
January 1992, 39-61.
[50] GAO, (July 1999). Best Practices: Better Management <strong>of</strong> <strong>Technology</strong> Can
Improve Weapon System Outcomes, GAO/NSIAD-99-162
[51] Smith, J.D. (2005) An Alternative to <strong>Technology</strong> Readiness Levels for Non-
Developmental Item (NDI) S<strong>of</strong>tware. Proceedings <strong>of</strong> the 38th Hawaii International
Conference on System Sciences
[52] National Audit Office (NAO). Accepting Equipment <strong>of</strong>f contract and into service,
Report by the Controller and Auditor General, MoD, 11 February 2000. [Online]
Available from:
http://www.nao.org.uk/publications/nao_reports/9900204.pdf [cited Dec 2006].
[53] Bruseberg, A. (2005). HFI-DTC. Cost-Justifying HFI (WP 3.7.2): A
comprehensive approach to provide evidence and guidance. (386731).
[54] RUSI Conference (Maritime Power in the 21st Century) 12/13 May 2004
(Whitehall) in West, R. and Shirley, D. Future Network Enabled Capability: barriers
to technology insertion. Journal <strong>of</strong> Defence Science, 9 No. 3.
[55] Marshall, P. and Underhill, G. (2004). Tornado Capability Lean –the enabler for
technology insertion. Journal <strong>of</strong> Defence Science, 9 No. 3.
[56] US Department <strong>of</strong> Defense, Chairman <strong>of</strong> the Joint Chiefs <strong>of</strong> Staff. Joint Vision
2020 (Washington: GPO, June 2000).
[57] Gentry, J.A. (2002). Doomed to Fail: America’s Blind Faith in Military
<strong>Technology</strong> Parameters, Winter, 03, 88-103.
[58] Langley, R.J. (1994). COTS Integration Issues, Risks, and Approaches, SIG
<strong>Technology</strong> Review, TRW Systems and Integration Group, 2, Winter, 4-14.
[59] Richardson, D. (2002). <strong>The</strong> COTS Revolution: Complete Guide. Armada
International, 5, 2002.
[60] Martin, M.D. and Rouncefield, M.D. (2005). That's How <strong>The</strong> Bastille Got
Stormed, Issues <strong>of</strong> Responsibility in User-Designer Relations, In Mackie, J.,
Rouncefield, M., (2005), (eds), Proceedings <strong>of</strong> the 5th Annual DIRC Research
Conference, 82-91.
[61] Jeffcott, M. and Johnson, C. (2005). <strong>The</strong> use <strong>of</strong> a formalised risk model in NHS
information system development. [Online] Available from:
http://www.dcs.gla.ac.uk/~johnson/papers/NHS_paper_CTW.pdf [cited Dec 2006].
70

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[62] Eason, K.D. (1988). Information <strong>Technology</strong> and Organisational Change.
London: Taylor Francis.
[63] Hornby, C., Clegg, C., Robson, J., McClaren, C., Richardson, S. and O’Brien, P.
(1992). Human and organisational issues in information systems development.
Behaviour and Information <strong>Technology</strong>, 11, 160-174.
[64] McKenzie, D. (1994). Computer-related accidental death: an empirical
exploration. Science and Public Policy, 21, 233-248.
[65] Skorczewski, L. (2004). <strong>Technology</strong> <strong>Insertion</strong> in Military Aerospace
Programmes.
Journal <strong>of</strong> Defence Science, 9 No. 3.
[66] Edmondson, A.C. Bohmer, R.M. and Pisano, G.P. (2001). Disrupted Routines:
Team learning and new technology implementation in hospitals. Administrative
Science Quarterly, Dec 2001.
[67] Keen, P. (1981). Information Systems and Organisational Change,
Communications <strong>of</strong> the ACM,24, 24-33.
[68] Tolbert, P.S. and Zucker, L.G. (1983). ‘Institutional Sources <strong>of</strong> Change in the
Formal Structure <strong>of</strong> Organisations: <strong>The</strong> Diffusion <strong>of</strong> Civil Service Reform, 1880-
1935’. Administrative Science Quarterly, 2, 22-39.
[69] Kim H.J. and Bretschneider, S. (2004). ‘Local Government Information
<strong>Technology</strong> Capacity: An Exploratory <strong>The</strong>ory’, Proceedings <strong>of</strong> the 37th Annual
Hawaii International Conference on System Sciences (HICSS '04) - Track 5.
[70] Nedovic-Budic, Z. (1998). <strong>The</strong> Likelihood <strong>of</strong> Becoming a GIS User. URISA
Journal, 10, 2.
[71] Brod, C. (1985). How to Deal with Technostress. In Managing New
Technologies: <strong>The</strong> In formation Revolution in Local Government. in Toregas, C.
(ed.), 70—74. Washington, DC: International City Management Association.
[72] Mohr, L.B. (1969). Determinants <strong>of</strong> Innovations in Organisations. American
Political Science Review, 63, 111—26.
[73] Peterson, C.M. and Peterson, L.O. (1988). <strong>The</strong> Dark Side <strong>of</strong> Office Automation:
How People Resist the Introduction <strong>of</strong> Office Automation <strong>Technology</strong>. In Human
Factors in Management Information Systems, Editor J. M. Carey Norwood, NJ:
ABLEX Publishing Corporation.
[74] Mitchell, S. (1994). Technophiles and Technophobes. American Demographics,
February, 36-42.
[75] Raghavan, S.A., and Chand, D.R. (1989). Diffusing S<strong>of</strong>tware Engineering
Methods. IEEE S<strong>of</strong>tware, July, 81—90.
[76] Carey, J.M., (ed.). (1988). Human Factors in Management Information Systems.
New Jersey: ABLEX Publishing Corporation.
71

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[77] Eason, K.D. (1993). Planning for Change: Introducing a Geographical
Information System. In Human Factors in Geographical In formation Systems, David
Medyckyj-Scott and Hilary M. Hearnshaw, (eds.), 199—210. London: Belhaven
Press.
[78] Rogers, E.M. (1962). Diffusion <strong>of</strong> Innovations. <strong>The</strong> Free Press, New York.
[79] Rogers E.M. (1983). Diffusion <strong>of</strong> Innovations. <strong>The</strong> Free Press, New York, 3rd
Edition.
[80] Moore, G.C. and Benbasat, I. (1991). Development <strong>of</strong> an instrument to measure
the perceptions <strong>of</strong> adopting an information technology innovation. Information
Systems Research, 2, 192-222.
[81] Ajzen, I. and Fishbein, M. (1980). Understanding attitudes and predicting social
behaviour. Eaglewood Cliffs, NJ: Prentice-Hall.
[82] Venkatesh, V., Morris, M.G., Davis, G.B., and Davis, F.D. (2003). User
acceptance <strong>of</strong> information technology: Toward a unified view. MIS Quarterly, 27,
425-478.
[83] Straub, D., Keil, M. and Brenner, W. (1997). Testing the technology acceptance
model across cultures: A three country study, Information & Management, 33, 1-11.
[84] Venkatesh, V. and Davis, F.D. (2000). A theoretical extension <strong>of</strong> the technology
acceptance model: Four longitudinal field studies. Management Science, 46, 425-478.
[85] Bagozzi, R.P., Davis, F.D., and Warshaw, P.R. (1992). Development and test <strong>of</strong> a
theory <strong>of</strong> technological learning and usage. Human Relations, Vol. 45, 660-686.
[86] Taylor, S. and Todd, P. A. (1995). Assessing IT usage: <strong>The</strong> role <strong>of</strong> prior
experience, MIS Quarterly, 19, 561-570.
[87] Venkatesh, V. and Morris, M.G. (2000). Why don't men ever stop to ask for
directions? Gender, social influence, and their role in technology acceptance and
usage behavior, MIS Quarterly, 24, 115-139.
[88] Sun, H., and Zhang, P. (2004). A Methodological Analysis <strong>of</strong> User <strong>Technology</strong>
Acceptance. Proceedings <strong>of</strong> the 37th Hawaii International Conference on System
Sciences.
[89] Van der Heijden, H. (2004). User acceptance <strong>of</strong> hedonic information systems,
MIS Quarterly, 28, 695-704.
[90] Sun, H. and Zhang, P. (2006). <strong>The</strong> role <strong>of</strong> moderating factors in user technology
acceptance, International Journal <strong>of</strong> Human-Computer Studies, 64, 53-78.
[91] Clarke, R. (1999). A Primer in Diffusion <strong>of</strong> Innovations <strong>The</strong>ory. Visiting Fellow,
Department <strong>of</strong> Computer Science, Australian National University Notes <strong>of</strong> May 1991,
revised May 1994, 26 September 1999.
72

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[92] Aarons, G.A. (2004). Mental health provider attitudes toward adoption <strong>of</strong>
evidence-based practice. <strong>The</strong> Evidence-Based Practice Attitude Scale (EBPAS).
Mental Health Services Research, 6, 61-74.
[93] Aarons, G.A. (2005). Measuring provider attitudes toward evidence-based
practice Organisational context and individual differences. Child and Adolescent
Psychiatric Clinics <strong>of</strong> North America, 14, 255-271.
[94] Glisson, C. and James, L. R. (2002). <strong>The</strong> cross-level effects <strong>of</strong> culture and climate
in human service teams. Journal <strong>of</strong> Organisational Behavior, 23, 767-794.
[95] Schoenwald, S.K., Letourneau, E.J., and Halliday-Boykins, C.A. (2004).
<strong>The</strong>rapist and client predictors <strong>of</strong> adherence to an evidence-based treatment in
community settings. Manuscript submitted for review.
[96] Mahnken, T.G. (2001). Naval War College Review, Summer 2001, Vol. LIV, No.
3.
[97] Donnelly, T. (2000). Revolution? What Revolution? Jane’s Defence Weekly, 7
June, 22–27.
[98] Leonard-Barton, D. and Deschamps, I. (1988). Managerial Influence in the
Implementation <strong>of</strong> New <strong>Technology</strong>. Management Science, 34, 1252-1266.
[99] Yin, R.K. (1977). Production efficiency versus bureaucratic self-interest: Two
innovative processes? Policy Sciences, 8, 381-399.
[100] Szulanski, G. (2000). <strong>The</strong> process <strong>of</strong> knowledge transfer: A diachronic analysis <strong>of</strong>
stickiness. Organisational Behavior and Human Decision Processes, 82, 9-27.
[101] Barley, S.R. (1986). <strong>Technology</strong> as an occasion for structuring: Evidence from
observations <strong>of</strong> CT scanners and the social order <strong>of</strong> radiology departments.
Administrative Science Quarterly, 31, 78-108.
[102] Attewell, P. (1992). <strong>Technology</strong> diffusion and organisational learning: <strong>The</strong> case
<strong>of</strong> business computing. Organisation Science, 3, 1-19.
[103] Orlikowski, W.J. (1993). Learning from notes: Organisational issues in groupware
implementation. Information Society, 9, 237-250.
[104] Orlikowski, W.J (2000). Using technology and constituting structures: A practice
lens for studying technology in organisations. Organisation Science, 11, 404-428.
[105] Moreland, R.L. (1999). Learning who knows what in work groups and
organisations. In L.L. Thompson, J.M. Levine, and D.M. Messick (eds.), Shared
Cognition in Organisations: <strong>The</strong> Management <strong>of</strong> Knowledge: 3-32. Mahwah, NJ:
Lawrence Erlbaum.
[106] Preece, J., Rogers, Y. and Benyon, D. (1994). Human Computer Interaction.
Wokingham, England, Reading. Addison-Wesley.
[107] Lehr, W. and Lichtenberg, F.R. Computer Use and Productivity Growth in
Federal Government Agencies 1987-92, Journal <strong>of</strong> Industrial Economics, 46, 257-
279.
73

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[108] Berger, W. (1999). Lost in space. Wired. [Online] Available from:
http://www.wired.com/wired/archive/7.02/chiat.html. [cited Dec 2006].
[109] Solow, R. <strong>The</strong> Productivity Paradox, Personal Communication, March 1990.
[110] Strassman, P.A. (1997). <strong>The</strong> Squandered Computer: Evaluating the Business
Alignment <strong>of</strong> Information Technologies. New Canadian, CT: Information Economics
Press.
[111] Brown, J. and Duguid, P. (2000). <strong>The</strong> Social Life <strong>of</strong> Information. Boston: Harvard
Business School Press.
[112] Mukhopadhyay, T., Rajiv, S. and Srinivasan, K. (1997). Information <strong>Technology</strong>
<strong>Impact</strong> on Process Output and Quality, Management Science, 43: 1645-1659.
[113] Brynjolfsson, E. and Hitt, L. (1996). Paradox Lost? Firm-level Evidence on the
Returns to Information Systems Spending. Management Science, April 1996.
[114] Dewan, S. and Min, C.K. (1997). Substitution <strong>of</strong> Information <strong>Technology</strong> for
Other Factors <strong>of</strong> Production: A Firm-level Analysis, Management Science 43, 1660-
1675.
[115] Bharadwaj, A.S., Bharadwaj, S.G., and Konsynski, B.R. (1999) Information
<strong>Technology</strong> Effects on Firm Performance as Measured by Tobin's Q. Journal <strong>of</strong>
Management Science.
[116] Devaraj, S. and Kohli, R. (2000). IT Pay<strong>of</strong>f in the Health Care Industry: A
Longitudinal Study. Journal <strong>of</strong> Management. Information Systems.
[117] Sohal, A.S., Moss, S. and Ng, L. (2001). Comparing IT success in manufacturing
and service industries. International Journal <strong>of</strong> Operations & Production
Management.
[118] David, P. (1990). <strong>The</strong> dynamo and the computer: An historical perspective on the
modern productivity paradox. American Economic Review, 80, 355-6
[119] Nielsen, J. (1994). Usability Engineering, Morgan Kaufmann Publishers,
[120] Bias R.G., and Mayhew, D. J, Eds. (1994). Cost-Justifying Usability, Harcourt
Brace & Co. Boston.
[121] Osterman, P. (1986). <strong>The</strong> <strong>Impact</strong> <strong>of</strong> Computers on the Employment <strong>of</strong> Clerks and
Managers. Industrial and Labor Relations Review, 39, 175-186.
[122] Morrison, C.J. and Berndt, E.R. (1990). Assessing the Productivity <strong>of</strong> Information
<strong>Technology</strong> Equipment in the U.S. Manufacturing Industries. National Bureau <strong>of</strong>
Economic Research Working Paper #3582 (January, 1990).
[123] CONGRESS OF THE UNITED STATES (1985). Automation <strong>of</strong> America’s
Offices 1985/2000 (Washington DC: Office <strong>of</strong> <strong>Technology</strong> Assessment). In Eason, K.
(2001) Changing perspectives on the organisational consequences <strong>of</strong> information
technology. Behaviour & Information <strong>Technology</strong>. 20, 323-328.
74

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[124] Woodward, J. (1958). Management and <strong>Technology</strong>. London: HMSO.
[125] Zub<strong>of</strong>f, S. (1988). In the Age <strong>of</strong> the Smart Machine. New York: Basic Books.
[126] Lohr, S. (1996). <strong>The</strong> network computer as the PC’s evil twin. New York Times, 4
November, sec. D p.1.
[127] Fukuyama, F. and Shulsky, A. (1997). <strong>The</strong> Virtual Corporation and Army
Organisation. Santa Monica, CA: Rand.
[128] Bjorn-Andersen N., Eason, K. and Robey, D. (1986). Managing Computer
<strong>Impact</strong>, Norwood NJ: Ablex Publishers.
[129] Langley, A. and Traux, J. (1994). A process study <strong>of</strong> new technology adoption in
smaller manufacturing firms. Journal <strong>of</strong> Management Studies, 31, 619-652.
[130] Kakihara, M. and Sørensen, C. (2002). Mobility: An Extended Perspective. In
Proceedings <strong>of</strong> the 35th Hawaii International Conference on System Sciences
(HICSS-35). IEEE, Big Island, Hawaii. 7th-10th January 2002
[131] Buchanan, D. and Boddy, D. (1983). Advanced <strong>Technology</strong> and the Quality <strong>of</strong>
Working Life: <strong>The</strong> Effects <strong>of</strong> Computerized Controls on Biscuit-making Operators,
Journal <strong>of</strong> Occupational Psychology, 56, 109-119.
[132] Buchanan, D.A. and Boddy, D. (1983). Organisations in the Computer Age:
Technological Imperatives and Strategic Choice. Gower.
[133] Patrickson, M. (1986). Adaptation by employees to new technology. Journal <strong>of</strong>
Occupational Psychology 59, 1-11, Cambridge University Press.
[134] House <strong>of</strong> Commons, Session 1992-93, Defence Committee Third Report – <strong>The</strong>
SA80 Rifle and Light Support Weapon. Report, together with the Proceedings <strong>of</strong> the
Committee relating to the Report. Minutes <strong>of</strong> Evidence and Memoranda, 9 June 1993.
<strong>The</strong> Stationery Office Books, ISBN 0100209831, 1993.
[135] Kimble, K. and McLoughlin, K. (1995). Computer Based Information Systems
and Manager's Work in New <strong>Technology</strong>, Work and Employment, 10, March,. 56 –
67.
[136] Englebart, D.C. (1995). Towards augmenting the human intellect and boosting
our collective IQ. Communications <strong>of</strong> the ACM, 38, 30-33.
[137] Jenkins, C. and Sherman, B. (1979). <strong>The</strong> Collapse <strong>of</strong> Work. London: Eyre
Methuen.
[138] Whisler, T.L. (1970). <strong>The</strong> <strong>Impact</strong> <strong>of</strong> Computers on Organisations. New York.
Praeger.
[139] Braverman, H. (1974). Labor and Monopoly Capital: <strong>The</strong> Degradation <strong>of</strong> Work in
the Twentieth Century, New York: Monthly Review Press.
[140] Buchanan, D. and Boddy, D. (1983). Organisations in the Computer Age:
Technological Imperatives and Strategic Choice, Gower.
75

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[141] Orlikowski, W. (1992). <strong>The</strong> Duality <strong>of</strong> <strong>Technology</strong>: Rethinking the Concept <strong>of</strong>
<strong>Technology</strong> in Organisations. Organisation Science, 3, 398-427.
[142] Chandler, D. (2000). Technological or Media Determinism. [Online] Available
from:
http://www.aber.ac.uk/ [Cited July 2007]
[143] Aldrich, H.E. (1972) <strong>Technology</strong> and Organisation Structure: A Reexamination <strong>of</strong>
the Findings <strong>of</strong> the Aston Group, Administrative Science Quarterly, 17, 26-43.
[144] Blau, P., McHugh-Falbe, C. McKinley, W. and Phelps, T. (1976). <strong>Technology</strong>
and Organisation in Manufacturing, Administrative Science Quarterly, 21, 20-40.
[145] Hickson, D., Pugh, D.S. and. Pheysey, D. (1969). Operations <strong>Technology</strong> and
Organisation Structure: An Empirical Reappraisal, Administrative Science Quarterly,
14, 378-397.
[146] Perrow, C.A. (1967). Framework for the Comparative Analysis <strong>of</strong> Organisations.
American Sociological Review.
[147] Shepard, J. (1977). <strong>Technology</strong>, Alienation, and Job Satisfaction, Annual Review
<strong>of</strong> Sociology, 3, 1-21.
[148] Carter, N.M. (1984). Computerization as a Predominate <strong>Technology</strong>: Its Influence
on the Structure <strong>of</strong> Newspaper Organisations, Academy <strong>of</strong> Management Journal, 27,
247-270.
[149] Foster, L.W. and Flynn D.M. (1984). Management Information <strong>Technology</strong>: Its
Effects on Organisational Form and Function, MIS Quarterly, 229-235.
[150] Hiltz, S.R. and. Johnson, K (1990). User Satisfaction with Computer-Mediated
Communication Systems, Management Science, 36, 739-764.
[151] Leavitt, H.J. and Whistler, T. L. (1958). Management in the 1980s, Harvard
Business Review, 36, 41-48.
[152] Pfeffer, J. and Leblebici, H. (1977). Information <strong>Technology</strong> in Organisational
Structure, Pacific Sociological Review, 20, 241-261.
[153] Siegel. J., Dubrovsky, V., Kiesler, S. and McGuire, T.W (1986). Group Processes
in Computer-Mediated Communication, Organisational Behavior and Human
Decision Processes, 37, 157-187.
[154] Giddens, A. (1984). <strong>The</strong> Constitution <strong>of</strong> Society: Outline <strong>of</strong> the <strong>The</strong>ory <strong>of</strong>
Structure, Berkeley CA: University <strong>of</strong> California Press.
[155] Jarvenpaa, S.L. (1989). Effects <strong>of</strong> Task Demands and Graphical Format on
Information Processing Strategies, Management Science, 35, 285-303.
[156] Feenberg, A. (2004). Democratic Rationalization. Readings in the Philosophy <strong>of</strong>
<strong>Technology</strong>. David M. Kaplan. Oxford: Rowman & Littlefield. 209-225
76

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[157] Bijker, W. E., Hughes, T.P. and Pinch, T. eds. (1987). <strong>The</strong> Social Construction <strong>of</strong>
Technological Systems: New Directions in the Sociology and History <strong>of</strong> <strong>Technology</strong>.
Cambridge, MA: MIT Press.
[158] Lucas, H.C. Jr. (1975). Performance and the use <strong>of</strong> an Information System,
Management Science, 20, 908-919.
[159] Child, J. (1972). Organisational Structure, Environment and Performance: <strong>The</strong>
Role <strong>of</strong> Strategic Choice, Sociology, 6, 1-22.
[160] Davis, L.E. and Taylor, J.C. (1986). <strong>Technology</strong>, Organisation and Job Structure,
in R. Dubin (Ed.), Handbook <strong>of</strong> Work, Organisation, and Society, Chicago IL: Rand
McNally, 379-419.
[161] Kling, R. and Iacono, S. (1984). Computing as an Occasion for Social Control.
Journal <strong>of</strong> Social Issues, 40, 77-96.
[162] Markus, M.L. (1983). Power, Politics, and MIS Implementation, Communications
<strong>of</strong> the ACM, 26, 430-444.
[163] Perrow, C. (1984). Normal accidents: Living with high-risk technologies. New
York: Basic Books.
[164] Trist, E.L., Higgin, G.W., Murray, H. and Pollock, A.B. (1963). Organisational
Choice, London. UK: Tavistock.
[165] Bostrom, R.P. and Heinen, J.S. (1977). MIS Problems and Failures: A Socio-
Technical Perspective, MIS Quarterly, 1, 11-28.
[166] Mumford, E. (1981). Participative Systems Design: Structure and Method,
Systems, Objectives, Solutions, 1, 5-19.
[167] McLoughlin, I. (1999). Creative Technological Change: <strong>The</strong> Shaping <strong>of</strong>
<strong>Technology</strong> and Organisations, Routledge, London.
[168] Grint, K. and Woolgar, S. (1997). <strong>The</strong> Machine at Work, Polity Press, Cambridge.
[169] MacKenzie, D.A. and Wajcman, J. (1985). <strong>The</strong> social shaping <strong>of</strong> technology: how
the refrigerator got its hum. Open University Press.
[170] Eason, K.D. (1997). Understanding the organisational ramifications <strong>of</strong>
implementing information technology systems. In Helander, M.G., Landauer, T.K.
and Prabhu, P.V. (eds), Handbook <strong>of</strong> Human-Computer Interaction (Amsterdam:
Elsevier), 1475-1495.
[171] Rose, J. and Jones, M. (2005). <strong>The</strong> Double Dance <strong>of</strong> Agency: A Socio-<strong>The</strong>oretic
Account <strong>of</strong> How Machines and Humans Interact. Systems, Signs & Actions, 1, 19–37.
[172] Orlikowski, W.I. and Iacono, C.S. (2001). Research Commentary: Desperately
Seeking the IT in IT Research, A Call to <strong>The</strong>orizing the IT Artefact, Information
Systems Research, Vol. 12, 121-134.
77

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[173] Attewell, P. (1998). Research on Information <strong>Technology</strong> <strong>Impact</strong>s, in Fostering
Research On <strong>The</strong> Economic and Social <strong>Impact</strong>s <strong>of</strong> Information <strong>Technology</strong>, 133, 134
(<strong>The</strong> National Research Council, 1998).
[174] Complexity theory as a way <strong>of</strong> understanding the web Speech at 45th Annual
Conference <strong>of</strong> the Society for Technical Communication, May, 1998. Proceedings <strong>of</strong>
the 45 th Annual Conference. Arlington, VA: Society for Technical Communication.
1998. 207-209.
[175] Scott, B. (2007). <strong>The</strong> Fractal Nature. Centre for Conscious Creativity. [Online]
Available from:
http://www.consciouscreativity.com [cited July 2007].
[176] Hayek, F. (1952). <strong>The</strong> Sensory Order: An Inquiry into the Foundations <strong>of</strong>
<strong>The</strong>oretical Psychology, <strong>The</strong> University <strong>of</strong> Chicago Press.
[177] Burnes, B. (1989). New <strong>Technology</strong> in Context. Aldershot: Gower.
[178] Blackler, F. and Brown, C. (1986). Alternative Models to guide the design and
introduction <strong>of</strong> new information technologies into work organisations. Journal <strong>of</strong>
Occupational Psychology, 59, 287-313.
[179] Keen, P. (1991). Shaping the Future. Harvard Business Press, Boston.
[180] Willcocks, L. (ed.) (1994). Information Management: <strong>The</strong> Evaluation <strong>of</strong>
Information Systems Investments. Chapman and Hall, London.
[181] Willcocks, L. (ed.) (1996). Investing in Information Systems: Evaluation and
Management. Chapman and Hall, London.
[182] Pike, R. and Ho, S. (1991). Risk Analysis in Capital Budgeting: Barriers and
Benefits. Omega 19, 235-245.
[183] Willcocks, L. (1993). Of Capital Importance: Evaluation and Management <strong>of</strong>
Information Systems Investments. Chapman and Hall, London.
[184] Beynon-Davies, P. (1995) Information systems ‘failure’: the case <strong>of</strong> the London
ambulance service’s computer aided despatch system. European Journal <strong>of</strong>
Information Systems, 4, 171–184.
[185] Beynon-Davis, P. (1999). Human error and information systems failure: the case
<strong>of</strong> the London Ambulance service computer-aided despatch system project.
Interacting with Computers. 11, 699-720.
[186] Lyytinen, K. and Hirscheim, R. (1987). Information systems failure: A survey and
classification <strong>of</strong> the empirical literature. Oxford Surveys in Information <strong>Technology</strong>,
4, 257-309.
[187] Pettigrew, A. and Whipp, R. (1991). Managing Change for Competitive Success.
Basil Blackwood, London.
[188] Pettigrew, A., Ferlie, E. and McKee, L. (1992). Shaping Strategic Change. Sage,
London.
78

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[189] Willcocks, L. and Margetts, H. (1994). Risk Assessment and Information
Systems. European Journal <strong>of</strong> Information Systems; 3, 127-138.
[190] Leveson, N. (1995). Safeware: System Safety and Computers: A Guide to
Preventing Accidents and Losses Caused by <strong>Technology</strong>. Addison Wesley, USA.
[191] Szulanski, G. (1996). Exploring internal stickiness: Impediments to the transfer <strong>of</strong>
best practice within the firm. Strategic Management Journal, 17, 27-43.
[192] Jorgensen, M and Sjoberg, D. (2000) <strong>The</strong> Importance <strong>of</strong> NOT Learning from
Experience. Readings from University <strong>of</strong> Maryland, College <strong>of</strong> Engineering.
[193] Gray, P.H.(2001). <strong>The</strong> impact <strong>of</strong> knowledge repositories on power and control in the
workplace. Information <strong>Technology</strong> and People, 14, pp 368-384
[194] Wilson, T.D. (2002). <strong>The</strong> nonsense <strong>of</strong> knowledge management. Information Research
[Online]. Available from:
http://informationr.net/ir/8-1/paper144.html [Cited August 2007]
[195] McEvily, S.K. and Chakvarthy, B. (2002) <strong>The</strong> Persistence <strong>of</strong> Knowledge-Based
Advantage: An Empirical Test for product Performance and Technological
Knowledge. Strategic Management Journal 23, 285-305.
[196] Glazer, R. (1998). Measuring the Knower: Towards a <strong>The</strong>ory <strong>of</strong> Knowledge
Equity. California Management Review 40, 175-194.
[197] Grover, V. and Davenport, T. (2001). General Perspectives on Knowledge
Management: Fostering a Research Agenda. Journal <strong>of</strong> Management Information
Systems 18, 5-22.
[198] Bontis, N. (2001). Assessing Knowledge Assets: A Review <strong>of</strong> the Models used to
Measure Intellectual Capital. International Journal <strong>of</strong> Management Review, 3, 41-60.
[199] MoD Procurement Development Group. Learning From Experience (LFE)
Information and Process Guide Version No. 2 January 2005 [Online]. Available
from:
http://www.ams.mod.uk/content/docs/lfeguide/lfeguide.pdf [cited Dec 2006].
[200] Gray, P.H. (2001). <strong>The</strong> impact <strong>of</strong> knowledge repositories on power and control in
the workplace. in Information <strong>Technology</strong> & People,14.
[201] Framework for knowledge management ICTD Project Newsletter
July-2007. In i4d (Information For Development) [Online]. Available from:
www.14donline.net [Cited July 2007]
[202] Harris, D. B. (1996). Creating a Knowledge Centric Information <strong>Technology</strong>
Environment, [Online] Available from:
http://www.htca.com/ckc.htm [cited Dec 2006].
79

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[203] Stata, R. (1989). Organizational Learning--<strong>The</strong> Key to Management Innovation,
Sloan Management Review, Spring, 63-74. Sensiper, S., AMS Knowledge Centers
Case N9-697-068, Boston: Harvard Business School.
[204] Davis, A. (2000). Transition from CRT to AMLCD on the Eur<strong>of</strong>ighter
Programme. In Cockpit Displays VII Displays for Defence Applications. Hopper,
D.G. Editor. Proceedings <strong>of</strong> SPIE Vol. 4022
[205] UK Warns USA Over ITAR Arms Restrictions. [Online] Available from:
http://www.defenseindustrydaily.com/2005/12/uk-warns-usa-over-itar-armsrestrictions/index.php
[cited Dec 2006].
[206] Military Health System loses nearly 5,000 records. [Online] Available from:
www.govhealthit.com Nov. 14, 2006. [cited Dec 2006].
[207] Dingwall, R. and Fenn, P. (1995). Risk management: Financial Implications. In:
Vincent, C. (ed.). Clinical risk management. London: BMJ Publishing Group.
[208] Evans, D. (1998). Where next for the hospital clinical claims manager? Clinical
Risk 1998; 4, 66-68.
[209] Page, D., Williams, P. and Boyd, D. (1993). Report <strong>of</strong> the public inquiry into the
London ambulance service, South West RHA.
[210] Eason, K.D. and Harker, S. (1989). An Open Systems Approach to Task Analysis,
HUSAT Research Centre, Loughborough University <strong>of</strong> <strong>Technology</strong>.
[211] Martin, B.H. and McLaughlin (2002). Personnel and Organisational Implications
<strong>of</strong> Project Falcon. QinetiQ Ltd.
[212] Bowyer, S.J. and Martin, B.H. (2003). Personnel and Organisational Implications
<strong>of</strong> FIST (UC). QinetiQ Ltd.
[213] Sandborn, P. and Singh, P. (2004). Forecasting <strong>Technology</strong> <strong>Insertion</strong> Concurrent
with Design Refresh Planning for COTS-Based Electronics Systems. Logistics
Spectrum, Vol. 38, 25-28.
[214] Sandborn, P., Herald, T., Houston, J. and Singh, P. (2003). Optimum technology
insertion into systems based on the assessment <strong>of</strong> viability, IEEE Trans. on
Components and Packaging Technologies, 26, 734-738.
[215] Use <strong>of</strong> Simulation Assisted Reliability Assessment to Qualify COTS PEMS 2002
IEEE Microelectronics Reliability and Qualification. Workshop December 3, 2002
CALCE Electronic Products and Systems Center University <strong>of</strong> Maryland, College
Park 20742 (301) 405-5323. [Online] Available from:
http://www.aero.org/conferences/mrqw/2002-papers/B_Hillman.pdf [cited Dec 2006].
[216] Bearden, D.A., Duclos, D.P., Barrera, M.J.; Mosher, T.J. and Lao, N.Y. (1997).
Methodology for conceptual remote sensing spacecraft technology: insertion analysis
balancing performance, cost, and risk. Proc. SPIE Vol. 3221, p. 536-549, Sensors,
Systems, and Next-Generation Satellites, Hiroyuki Fujisada; Ed. 12, 1997.
80

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
[217] Luxhøj, J.T. and Kauffeld, K. (2003). Evaluating the Effect <strong>of</strong> <strong>Technology</strong>
<strong>Insertion</strong> into the National Airspace System. <strong>The</strong> Rutgers Scholar, Vol. 5.
[218] Luxhøj, J.T. and Maurino, M. (2001). An Aviation System Risk Model (ASRM)
Case Study: Air Ontario 1363, <strong>The</strong> Rutgers Scholar, Vol. 1.
[219] Tuckman, B.W. and Jensen, M.A.C. (1977). Stages <strong>of</strong> Small Group Development
Revisited, Group and Organization Studies, 2, 419–427.
[220] Noble, D. (2004). Understanding and Applying the Cognitive Foundation <strong>of</strong>
Effective Teamwork, Evidence Based Research Inc.
[221] Mathieson, G.L., Mistry, B. and Waters, M. (2005). Coping with Social and
Cultural Variables in C2 Modelling for Networked Enabled Forces. 10th
International Command and Control Research and <strong>Technology</strong> Symposium, VA,
USA.
[222] Mathieson, G.L., Ling, M.F. (2005). Evaluating the Effectiveness <strong>of</strong> an Agile,
Coalition, Network-enabled Force [Online] Available from:
www.dsto.defence.gov.au [cited Dec 2006].
[223] Fenton, N. and Neil, M. Measuring your Risks: Numbers that would make sense
to Bruce Willis and his crew. Whitepaper. [Online] Available from:
http://www.agenarisk.com/resources/white_papers/Measuring_Risks.pdf [cited Jul 2007]
[224] Wickens, D., Gordon, S. and Liu, Y. (1997). An Introduction to Human Factors
Engineering. Longman, New York.
[225] Willcocks, L. and Margetts, H. (1994). Informatization in Public and Private
Sector Settings: Distinctive or Common Risks Informatization and the Public Sector.
[226] Willcocks, L. and Mark, A. (1989). IT Systems Implementation: Research
Findings from the Public Sector, Journal <strong>of</strong> Information <strong>Technology</strong>, 4, 92-103.
[227] Brynjolfsson, E. (1993). <strong>The</strong> Productivity Paradox <strong>of</strong> Information <strong>Technology</strong>.
Communications <strong>of</strong> the ACM.
[228] Daft, R.L. (2007). Understanding the <strong>The</strong>ory and Design <strong>of</strong> Organisation.
Thomson, South Western.
[229] Bennett, L., Kinloch, A., McDermid, J. et al. (2004). Key Issues for Effective
<strong>Technology</strong> <strong>Insertion</strong>. Journal <strong>of</strong> Defence Science, 9, 3.
[230] Rogers, E.M. (1995). Attributes <strong>of</strong> Innovations and <strong>The</strong>ir Rate <strong>of</strong> Adoption.
Diffusion <strong>of</strong> Innovations. New York, NY: Simon & Schuster.
[231] Frambach, R.T. and Schillewaert, N. (2002). Organisational innovation adoption:
A multi-level framework <strong>of</strong> determinants and opportunities for future research.
Journal <strong>of</strong> Business Research, 55, 163-176.
81

HFIDTC/2/12.2.1/1
Version 3 / 21 November 2007
- End <strong>of</strong> Document -
82